diff options
| -rw-r--r-- | doc/api/errors.md | 10 | ||||
| -rw-r--r-- | doc/api/zlib.md | 212 | ||||
| -rw-r--r-- | tools/doc/type-parser.js | 2 |
3 files changed, 211 insertions, 13 deletions
diff --git a/doc/api/errors.md b/doc/api/errors.md index 9d6f438508..2b4c1ef375 100644 --- a/doc/api/errors.md +++ b/doc/api/errors.md @@ -624,16 +624,16 @@ An attempt was made to register something that is not a function as an The type of an asynchronous resource was invalid. Note that users are also able to define their own types if using the public embedder API. +<a id="ERR_BROTLI_COMPRESSION_FAILED"></a> +### ERR_BROTLI_COMPRESSION_FAILED + +Data passed to a Brotli stream was not successfully compressed. + <a id="ERR_BROTLI_INVALID_PARAM"></a> ### ERR_BROTLI_INVALID_PARAM An invalid parameter key was passed during construction of a Brotli stream. -<a id="ERR_BROTLI_COMPRESSION_FAILED"> -### ERR_BROTLI_COMPRESSION_FAILED - -Data passed to a Brotli stream was not successfully compressed. - <a id="ERR_BUFFER_CONTEXT_NOT_AVAILABLE"></a> ### ERR_BUFFER_CONTEXT_NOT_AVAILABLE diff --git a/doc/api/zlib.md b/doc/api/zlib.md index 7b8f9187ef..cbf3ebd180 100644 --- a/doc/api/zlib.md +++ b/doc/api/zlib.md @@ -5,7 +5,7 @@ > Stability: 2 - Stable The `zlib` module provides compression functionality implemented using Gzip and -Deflate/Inflate. It can be accessed using: +Deflate/Inflate, as well as Brotli. It can be accessed using: ```js const zlib = require('zlib'); @@ -54,8 +54,8 @@ and/or unrecoverable and catastrophic memory fragmentation. ## Compressing HTTP requests and responses -The `zlib` module can be used to implement support for the `gzip` and `deflate` -content-encoding mechanisms defined by +The `zlib` module can be used to implement support for the `gzip`, `deflate` +and `br` content-encoding mechanisms defined by [HTTP](https://tools.ietf.org/html/rfc7230#section-4.2). The HTTP [`Accept-Encoding`][] header is used within an http request to identify @@ -76,12 +76,15 @@ const fs = require('fs'); const request = http.get({ host: 'example.com', path: '/', port: 80, - headers: { 'Accept-Encoding': 'gzip,deflate' } }); + headers: { 'Accept-Encoding': 'br,gzip,deflate' } }); request.on('response', (response) => { const output = fs.createWriteStream('example.com_index.html'); switch (response.headers['content-encoding']) { - // Or, just use zlib.createUnzip() to handle both cases + case 'br': + response.pipe(zlib.createBrotliDecompress()).pipe(output); + break; + // Or, just use zlib.createUnzip() to handle both of the following cases: case 'gzip': response.pipe(zlib.createGunzip()).pipe(output); break; @@ -117,6 +120,9 @@ http.createServer((request, response) => { } else if (/\bgzip\b/.test(acceptEncoding)) { response.writeHead(200, { 'Content-Encoding': 'gzip' }); raw.pipe(zlib.createGzip()).pipe(response); + } else if (/\bbr\b/.test(acceptEncoding)) { + response.writeHead(200, { 'Content-Encoding': 'br' }); + raw.pipe(zlib.createBrotliCompress()).pipe(response); } else { response.writeHead(200, {}); raw.pipe(response); @@ -136,6 +142,7 @@ const buffer = Buffer.from('eJzT0yMA', 'base64'); zlib.unzip( buffer, + // For Brotli, the equivalent is zlib.constants.BROTLI_OPERATION_FLUSH. { finishFlush: zlib.constants.Z_SYNC_FLUSH }, (err, buffer) => { if (!err) { @@ -156,6 +163,8 @@ decompressed result is valid. <!--type=misc--> +### For zlib-based streams + From `zlib/zconf.h`, modified to Node.js's usage: The memory requirements for deflate are (in bytes): @@ -194,6 +203,16 @@ fewer calls to `zlib` because it will be able to process more data on each `write` operation. So, this is another factor that affects the speed, at the cost of memory usage. +### For Brotli-based streams + +There are equivalents to the zlib options for Brotli-based streams, although +these options have different ranges than the zlib ones: + +- zlib’s `level` option matches Brotli’s `BROTLI_PARAM_QUALITY` option. +- zlib’s `windowBits` option matches Brotli’s `BROTLI_PARAM_LGWIN` option. + +See [below][Brotli parameters] for more details on Brotli-specific options. + ## Flushing Calling [`.flush()`][] on a compression stream will make `zlib` return as much @@ -231,6 +250,8 @@ added: v0.5.8 <!--type=misc--> +### zlib constants + All of the constants defined in `zlib.h` are also defined on `require('zlib').constants`. In the normal course of operations, it will not be necessary to use these constants. They are documented so that their presence is @@ -281,6 +302,77 @@ Compression strategy. * `zlib.constants.Z_FIXED` * `zlib.constants.Z_DEFAULT_STRATEGY` +### Brotli constants +<!-- YAML +added: REPLACEME +--> + +There are several options and other constants available for Brotli-based +streams: + +#### Flush operations + +The following values are valid flush operations for Brotli-based streams: + +* `zlib.constants.BROTLI_OPERATION_PROCESS` (default for all operations) +* `zlib.constants.BROTLI_OPERATION_FLUSH` (default when calling `.flush()`) +* `zlib.constants.BROTLI_OPERATION_FINISH` (default for the last chunk) +* `zlib.constants.BROTLI_OPERATION_EMIT_METADATA` + * This particular operation may be hard to use in a Node.js context, + as the streaming layer makes it hard to know which data will end up + in this frame. Also, there is currently no way to consume this data through + the Node.js API. + +#### Compressor options + +There are several options that can be set on Brotli encoders, affecting +compression efficiency and speed. Both the keys and the values can be accessed +as properties of the `zlib.constants` object. + +The most important options are: + +* `BROTLI_PARAM_MODE` + * `BROTLI_MODE_GENERIC` (default) + * `BROTLI_MODE_TEXT`, adjusted for UTF-8 text + * `BROTLI_MODE_FONT`, adjusted for WOFF 2.0 fonts +* `BROTLI_PARAM_QUALITY` + * Ranges from `BROTLI_MIN_QUALITY` to `BROTLI_MAX_QUALITY`, + with a default of `BROTLI_DEFAULT_QUALITY`. +* `BROTLI_PARAM_SIZE_HINT` + * Integer value representing the expected input size; + defaults to `0` for an unknown input size. + +The following flags can be set for advanced control over the compression +algorithm and memory usage tuning: + +* `BROTLI_PARAM_LGWIN` + * Ranges from `BROTLI_MIN_WINDOW_BITS` to `BROTLI_MAX_WINDOW_BITS`, + with a default of `BROTLI_DEFAULT_WINDOW`, or up to + `BROTLI_LARGE_MAX_WINDOW_BITS` if the `BROTLI_PARAM_LARGE_WINDOW` flag + is set. +* `BROTLI_PARAM_LGBLOCK` + * Ranges from `BROTLI_MIN_INPUT_BLOCK_BITS` to `BROTLI_MAX_INPUT_BLOCK_BITS`. +* `BROTLI_PARAM_DISABLE_LITERAL_CONTEXT_MODELING` + * Boolean flag that decreases compression ratio in favour of + decompression speed. +* `BROTLI_PARAM_LARGE_WINDOW` + * Boolean flag enabling “Large Window Brotli” mode (not compatible with the + Brotli format as standardized in [RFC 7932][]). +* `BROTLI_PARAM_NPOSTFIX` + * Ranges from `0` to `BROTLI_MAX_NPOSTFIX`. +* `BROTLI_PARAM_NDIRECT` + * Ranges from `0` to `15 << NPOSTFIX` in steps of `1 << NPOSTFIX`. + +#### Decompressor options + +These advanced options are available for controlling decompression: + +* `BROTLI_DECODER_PARAM_DISABLE_RING_BUFFER_REALLOCATION` + * Boolean flag that affects internal memory allocation patterns. +* `BROTLI_DECODER_PARAM_LARGE_WINDOW` + * Boolean flag enabling “Large Window Brotli” mode (not compatible with the + Brotli format as standardized in [RFC 7932][]). + ## Class: Options <!-- YAML added: v0.11.1 @@ -298,7 +390,7 @@ changes: <!--type=misc--> -Each class takes an `options` object. All options are optional. +Each zlib-based class takes an `options` object. All options are optional. Note that some options are only relevant when compressing, and are ignored by the decompression classes. @@ -317,6 +409,47 @@ ignored by the decompression classes. See the description of `deflateInit2` and `inflateInit2` at <https://zlib.net/manual.html#Advanced> for more information on these. +## Class: BrotliOptions +<!-- YAML +added: REPLACEME +--> + +<!--type=misc--> + +Each Brotli-based class takes an `options` object. All options are optional. + +* `flush` {integer} **Default:** `zlib.constants.BROTLI_OPERATION_PROCESS` +* `finishFlush` {integer} **Default:** `zlib.constants.BROTLI_OPERATION_FINISH` +* `chunkSize` {integer} **Default:** `16 * 1024` +* `params` {Object} Key-value object containing indexed [Brotli parameters][]. + +For example: + +```js +const stream = zlib.createBrotliCompress({ + chunkSize: 32 * 1024, + params: { + [zlib.constants.BROTLI_PARAM_MODE]: zlib.constants.BROTLI_MODE_TEXT, + [zlib.constants.BROTLI_PARAM_QUALITY]: 4, + [zlib.constants.BROTLI_PARAM_SIZE_HINT]: fs.statSync(inputFile).size + } +}); +``` + +## Class: zlib.BrotliCompress +<!-- YAML +added: REPLACEME +--> + +Compress data using the Brotli algorithm. + +## Class: zlib.BrotliDecompress +<!-- YAML +added: REPLACEME +--> + +Decompress data using the Brotli algorithm. + ## Class: zlib.Deflate <!-- YAML added: v0.5.8 @@ -389,9 +522,13 @@ added: v0.5.8 Decompress either a Gzip- or Deflate-compressed stream by auto-detecting the header. -## Class: zlib.Zlib +## Class: zlib.ZlibBase <!-- YAML added: v0.5.8 +changes: + - version: REPLACEME + pr-url: https://github.com/nodejs/node/pull/24939 + description: This class was renamed from `Zlib` to `ZlibBase`. --> Not exported by the `zlib` module. It is documented here because it is the base @@ -440,7 +577,8 @@ Close the underlying handle. added: v0.5.8 --> -* `kind` **Default:** `zlib.constants.Z_FULL_FLUSH` +* `kind` **Default:** `zlib.constants.Z_FULL_FLUSH` for zlib-based streams, + `zlib.constants.BROTLI_OPERATION_FLUSH` for Brotli-based streams. * `callback` {Function} Flush pending data. Don't call this frivolously, premature flushes negatively @@ -460,6 +598,8 @@ added: v0.11.4 * `strategy` {integer} * `callback` {Function} +This function is only available for zlib-based streams, i.e. not Brotli. + Dynamically update the compression level and compression strategy. Only applicable to deflate algorithm. @@ -478,6 +618,24 @@ added: v7.0.0 Provides an object enumerating Zlib-related constants. +## zlib.createBrotliCompress([options]) +<!-- YAML +added: REPLACEME +--> + +* `options` {brotli options} + +Creates and returns a new [`BrotliCompress`][] object. + +## zlib.createBrotliDecompress([options]) +<!-- YAML +added: REPLACEME +--> + +* `options` {brotli options} + +Creates and returns a new [`BrotliDecompress`][] object. + ## zlib.createDeflate([options]) <!-- YAML added: v0.5.8 @@ -560,6 +718,40 @@ with `callback(error, result)`. Every method has a `*Sync` counterpart, which accept the same arguments, but without a callback. +### zlib.brotliCompress(buffer[, options], callback) +<!-- YAML +added: REPLACEME +--> +* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string} +* `options` {brotli options} +* `callback` {Function} + +### zlib.brotliCompressSync(buffer[, options]) +<!-- YAML +added: REPLACEME +--> +* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string} +* `options` {brotli options} + +Compress a chunk of data with [`BrotliCompress`][]. + +### zlib.brotliDecompress(buffer[, options], callback) +<!-- YAML +added: REPLACEME +--> +* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string} +* `options` {brotli options} +* `callback` {Function} + +### zlib.brotliDecompressSync(buffer[, options]) +<!-- YAML +added: REPLACEME +--> +* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string} +* `options` {brotli options} + +Decompress a chunk of data with [`BrotliDecompress`][]. + ### zlib.deflate(buffer[, options], callback) <!-- YAML added: v0.6.0 @@ -832,6 +1024,8 @@ Decompress a chunk of data with [`Unzip`][]. [`.flush()`]: #zlib_zlib_flush_kind_callback [`Accept-Encoding`]: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.3 [`ArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer +[`BrotliCompress`]: #zlib_class_zlib_brotlicompress +[`BrotliDecompress`]: #zlib_class_zlib_brotlidecompress [`Buffer`]: buffer.html#buffer_class_buffer [`Content-Encoding`]: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11 [`DataView`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView @@ -845,6 +1039,8 @@ Decompress a chunk of data with [`Unzip`][]. [`Unzip`]: #zlib_class_zlib_unzip [`stream.Transform`]: stream.html#stream_class_stream_transform [`zlib.bytesWritten`]: #zlib_zlib_byteswritten +[Brotli parameters]: #zlib_brotli_constants [Memory Usage Tuning]: #zlib_memory_usage_tuning +[RFC 7932]: https://www.rfc-editor.org/rfc/rfc7932.txt [pool size]: cli.html#cli_uv_threadpool_size_size [zlib documentation]: https://zlib.net/manual.html#Constants diff --git a/tools/doc/type-parser.js b/tools/doc/type-parser.js index e358885620..44ebb0b572 100644 --- a/tools/doc/type-parser.js +++ b/tools/doc/type-parser.js @@ -38,6 +38,8 @@ const customTypesMap = { 'AsyncHook': 'async_hooks.html#async_hooks_async_hooks_createhook_callbacks', 'AsyncResource': 'async_hooks.html#async_hooks_class_asyncresource', + 'brotli options': 'zlib.html#zlib_class_brotlioptions', + 'Buffer': 'buffer.html#buffer_class_buffer', 'ChildProcess': 'child_process.html#child_process_class_childprocess', |