diff options
author | Sam Roberts <vieuxtech@gmail.com> | 2017-08-23 14:59:06 -0700 |
---|---|---|
committer | Sam Roberts <vieuxtech@gmail.com> | 2017-08-25 10:16:00 -0700 |
commit | a1d34b3f491ce8e9c34cbc5ddf4e9052b0ed0a01 (patch) | |
tree | 37aa944f0d2041cd30d96f9edd7ab9ecd167d44f | |
parent | 6cdffb18c55743c1d20b5abc50fbe9d607e00cf1 (diff) | |
download | android-node-v8-a1d34b3f491ce8e9c34cbc5ddf4e9052b0ed0a01.tar.gz android-node-v8-a1d34b3f491ce8e9c34cbc5ddf4e9052b0ed0a01.tar.bz2 android-node-v8-a1d34b3f491ce8e9c34cbc5ddf4e9052b0ed0a01.zip |
doc: threadpool size, and APIs using the pool
Not knowing which APIs use libuv's threadpool can lead to surprising
performance problems. Document the APIs, and also document
UV_THREADPOOL_SIZE, which can be used to fix problems.
PR-URL: https://github.com/nodejs/node/pull/14995
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
-rw-r--r-- | doc/api/cli.md | 25 | ||||
-rw-r--r-- | doc/api/crypto.md | 13 | ||||
-rw-r--r-- | doc/api/dns.md | 22 | ||||
-rw-r--r-- | doc/api/fs.md | 8 | ||||
-rw-r--r-- | doc/api/zlib.md | 7 |
5 files changed, 66 insertions, 9 deletions
diff --git a/doc/api/cli.md b/doc/api/cli.md index 7463f2dfb3..beceebec77 100644 --- a/doc/api/cli.md +++ b/doc/api/cli.md @@ -568,6 +568,30 @@ appended to if it does. If an error occurs while attempting to write the warning to the file, the warning will be written to stderr instead. This is equivalent to using the `--redirect-warnings=file` command-line flag. +### `UV_THREADPOOL_SIZE=size` + +Set the number of threads used in libuv's threadpool to `size` threads. + +Asynchronous system APIs are used by Node.js whenever possible, but where they +do not exist, libuv's threadpool is used to create asynchronous node APIs based +on synchronous system APIs. Node.js APIs that use the threadpool are: + +- all `fs` APIs, other than the file watcher APIs and those that are explicitly + synchronous +- `crypto.pbkdf2()` +- `crypto.randomBytes()`, unless it is used without a callback +- `crypto.randomFill()` +- `dns.lookup()` +- all `zlib` APIs, other than those that are explicitly synchronous + +Because libuv's threadpool has a fixed size, it means that if for whatever +reason any of these APIs takes a long time, other (seemingly unrelated) APIs +that run in libuv's threadpool will experience degraded performance. In order to +mitigate this issue, one potential solution is to increase the size of libuv's +threadpool by setting the `'UV_THREADPOOL_SIZE'` environment variable to a value +greater than `4` (its current default value). For more information, see the +[libuv threadpool documentation][]. + [`--openssl-config`]: #cli_openssl_config_file [Buffer]: buffer.html#buffer_buffer [Chrome Debugging Protocol]: https://chromedevtools.github.io/debugger-protocol-viewer @@ -575,3 +599,4 @@ equivalent to using the `--redirect-warnings=file` command-line flag. [SlowBuffer]: buffer.html#buffer_class_slowbuffer [debugger]: debugger.html [emit_warning]: process.html#process_process_emitwarning_warning_type_code_ctor +[libuv threadpool documentation]: http://docs.libuv.org/en/latest/threadpool.html diff --git a/doc/api/crypto.md b/doc/api/crypto.md index 2f981e1ffe..05e0245086 100644 --- a/doc/api/crypto.md +++ b/doc/api/crypto.md @@ -1559,6 +1559,10 @@ crypto.pbkdf2('secret', 'salt', 100000, 512, 'sha512', (err, derivedKey) => { An array of supported digest functions can be retrieved using [`crypto.getHashes()`][]. +Note that this API uses libuv's threadpool, which can have surprising and +negative performance implications for some applications, see the +[`UV_THREADPOOL_SIZE`][] documentation for more information. + ### crypto.pbkdf2Sync(password, salt, iterations, keylen, digest) <!-- YAML added: v0.9.3 @@ -1723,6 +1727,10 @@ This should normally never take longer than a few milliseconds. The only time when generating the random bytes may conceivably block for a longer period of time is right after boot, when the whole system is still low on entropy. +Note that this API uses libuv's threadpool, which can have surprising and +negative performance implications for some applications, see the +[`UV_THREADPOOL_SIZE`][] documentation for more information. + ### crypto.randomFillSync(buffer[, offset][, size]) <!-- YAML added: v7.10.0 @@ -1783,6 +1791,10 @@ crypto.randomFill(buf, 5, 5, (err, buf) => { }); ``` +Note that this API uses libuv's threadpool, which can have surprising and +negative performance implications for some applications, see the +[`UV_THREADPOOL_SIZE`][] documentation for more information. + ### crypto.setEngine(engine[, flags]) <!-- YAML added: v0.11.11 @@ -2208,6 +2220,7 @@ the `crypto`, `tls`, and `https` modules and are generally specific to OpenSSL. [`Buffer`]: buffer.html [`EVP_BytesToKey`]: https://www.openssl.org/docs/man1.0.2/crypto/EVP_BytesToKey.html +[`UV_THREADPOOL_SIZE`]: cli.html#cli_uv_threadpool_size_size [`cipher.final()`]: #crypto_cipher_final_outputencoding [`cipher.update()`]: #crypto_cipher_update_data_inputencoding_outputencoding [`crypto.createCipher()`]: #crypto_crypto_createcipher_algorithm_password diff --git a/doc/api/dns.md b/doc/api/dns.md index be89447368..4248bdf43a 100644 --- a/doc/api/dns.md +++ b/doc/api/dns.md @@ -620,15 +620,16 @@ but note that changing these files will change the behavior of _all other programs running on the same operating system_. Though the call to `dns.lookup()` will be asynchronous from JavaScript's -perspective, it is implemented as a synchronous call to getaddrinfo(3) that -runs on libuv's threadpool. Because libuv's threadpool has a fixed size, it -means that if for whatever reason the call to getaddrinfo(3) takes a long -time, other operations that could run on libuv's threadpool (such as filesystem -operations) will experience degraded performance. In order to mitigate this -issue, one potential solution is to increase the size of libuv's threadpool by -setting the `'UV_THREADPOOL_SIZE'` environment variable to a value greater than -`4` (its current default value). For more information on libuv's threadpool, see -[the official libuv documentation][]. +perspective, it is implemented as a synchronous call to getaddrinfo(3) that runs +on libuv's threadpool. This can have surprising negative performance +implications for some applications, see the [`UV_THREADPOOL_SIZE`][] +documentation for more information. + +Note that various networking APIs will call `dns.lookup()` internally to resolve +host names. If that is an issue, consider resolving the hostname to and address +using `dns.resolve()` and using the address instead of a host name. Also, some +networking APIs (such as [`socket.connect()`][] and [`dgram.createSocket()`][]) +allow the default resolver, `dns.lookup()`, to be replaced. ### `dns.resolve()`, `dns.resolve*()` and `dns.reverse()` @@ -644,6 +645,8 @@ They do not use the same set of configuration files than what [`dns.lookup()`][] uses. For instance, _they do not use the configuration from `/etc/hosts`_. [`Error`]: errors.html#errors_class_error +[`UV_THREADPOOL_SIZE`]: cli.html#cli_uv_threadpool_size_size +[`dgram.createSocket()`]: dgram.html#dgram_dgram_createsocket_options_callback [`dns.getServers()`]: #dns_dns_getservers [`dns.lookup()`]: #dns_dns_lookup_hostname_options_callback [`dns.resolve()`]: #dns_dns_resolve_hostname_rrtype_callback @@ -660,6 +663,7 @@ uses. For instance, _they do not use the configuration from `/etc/hosts`_. [`dns.resolveTxt()`]: #dns_dns_resolvetxt_hostname_callback [`dns.reverse()`]: #dns_dns_reverse_ip_callback [`dns.setServers()`]: #dns_dns_setservers_servers +[`socket.connect()`]: net.html#net_socket_connect_options_connectlistener [`util.promisify()`]: util.html#util_util_promisify_original [DNS error codes]: #dns_error_codes [Implementation considerations section]: #dns_implementation_considerations diff --git a/doc/api/fs.md b/doc/api/fs.md index 9985858aca..929cf6941e 100644 --- a/doc/api/fs.md +++ b/doc/api/fs.md @@ -100,6 +100,13 @@ example `fs.readdirSync('c:\\')` can potentially return a different result than `fs.readdirSync('c:')`. For more information, see [this MSDN page][MSDN-Rel-Path]. +## Threadpool Usage + +Note that all file system APIs except `fs.FSWatcher()` and those that are +explicitly synchronous use libuv's threadpool, which can have surprising and +negative performance implications for some applications, see the +[`UV_THREADPOOL_SIZE`][] documentation for more information. + ## WHATWG URL object support <!-- YAML added: v7.6.0 @@ -2845,6 +2852,7 @@ The following constants are meant for use with the [`fs.Stats`][] object's [`ReadDirectoryChangesW`]: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365465%28v=vs.85%29.aspx [`ReadStream`]: #fs_class_fs_readstream [`URL`]: url.html#url_the_whatwg_url_api +[`UV_THREADPOOL_SIZE`]: cli.html#cli_uv_threadpool_size_size [`WriteStream`]: #fs_class_fs_writestream [`event ports`]: http://illumos.org/man/port_create [`fs.FSWatcher`]: #fs_class_fs_fswatcher diff --git a/doc/api/zlib.md b/doc/api/zlib.md index 8b5653a72c..0f58979949 100644 --- a/doc/api/zlib.md +++ b/doc/api/zlib.md @@ -43,6 +43,13 @@ zlib.unzip(buffer, (err, buffer) => { }); ``` +## Threadpool Usage + +Note that all zlib APIs except those that are explicitly synchronous use libuv's +threadpool, which can have surprising and negative performance implications for +some applications, see the [`UV_THREADPOOL_SIZE`][] documentation for more +information. + ## Compressing HTTP requests and responses The `zlib` module can be used to implement support for the `gzip` and `deflate` |