fs: introduce `opendir()` and `fs.Dir`

This adds long-requested methods for asynchronously interacting and
iterating through directory entries by using `uv_fs_opendir`,
`uv_fs_readdir`, and `uv_fs_closedir`.

`fs.opendir()` and friends return an `fs.Dir`, which contains methods
for doing reads and cleanup. `fs.Dir` also has the async iterator
symbol exposed.

The `read()` method and friends only return `fs.Dirent`s for this API.
Having a entry type or doing a `stat` call is deemed to be necessary in
the majority of cases, so just returning dirents seems like the logical
choice for a new api.

Reading when there are no more entries returns `null` instead of a
dirent. However the async iterator hides that (and does automatic
cleanup).

The code lives in separate files from the rest of fs, this is done
partially to prevent over-pollution of those (already very large)
files, but also in the case of js allows loading into `fsPromises`.

Due to async_hooks, this introduces a new handle type of `DIRHANDLE`.

This PR does not attempt to make complete optimization of
this feature. Notable future improvements include:
- Moving promise work into C++ land like FileHandle.
- Possibly adding `readv()` to do multi-entry directory reads.
- Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation.

Refs: https://github.com/nodejs/node-v0.x-archive/issues/388
Refs: https://github.com/nodejs/node/issues/583
Refs: https://github.com/libuv/libuv/pull/2057

PR-URL: https://github.com/nodejs/node/pull/29349
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: David Carlier <devnexen@gmail.com>

2 files changed, 220 insertions, 2 deletions

diff --git a/doc/api/errors.md b/doc/api/errors.md
index e13e25827e..8c86012630 100644
--- a/doc/api/errors.md
+++ b/doc/api/errors.md
@@ -826,6 +826,11 @@ A signing `key` was not provided to the [`sign.sign()`][] method.
 [`crypto.timingSafeEqual()`][] was called with `Buffer`, `TypedArray`, or
 `DataView` arguments of different lengths.
 
+<a id="ERR_DIR_CLOSED"></a>
+### ERR_DIR_CLOSED
+
+The [`fs.Dir`][] was previously closed.
+
 <a id="ERR_DNS_SET_SERVERS_FAILED"></a>
 ### ERR_DNS_SET_SERVERS_FAILED
 
@@ -2388,6 +2393,7 @@ such as `process.stdout.on('data')`.
 [`dgram.disconnect()`]: dgram.html#dgram_socket_disconnect
 [`dgram.remoteAddress()`]: dgram.html#dgram_socket_remoteaddress
 [`errno`(3) man page]: http://man7.org/linux/man-pages/man3/errno.3.html
+[`fs.Dir`]: fs.html#fs_class_fs_dir
 [`fs.readFileSync`]: fs.html#fs_fs_readfilesync_path_options
 [`fs.readdir`]: fs.html#fs_fs_readdir_path_options_callback
 [`fs.symlink()`]: fs.html#fs_fs_symlink_target_path_type_callback
diff --git a/doc/api/fs.md b/doc/api/fs.md
index c22e169f1c..a6736d127e 100644
--- a/doc/api/fs.md
+++ b/doc/api/fs.md
@@ -284,13 +284,148 @@ synchronous use libuv's threadpool, which can have surprising and negative
 performance implications for some applications. See the
 [`UV_THREADPOOL_SIZE`][] documentation for more information.
 
+## Class fs.Dir
+<!-- YAML
+added: REPLACEME
+-->
+
+A class representing a directory stream.
+
+Created by [`fs.opendir()`][], [`fs.opendirSync()`][], or [`fsPromises.opendir()`][].
+
+Example using async interation:
+
+```js
+const fs = require('fs');
+
+async function print(path) {
+  const dir = await fs.promises.opendir(path);
+  for await (const dirent of dir) {
+    console.log(dirent.name);
+  }
+}
+print('./').catch(console.error);
+```
+
+### dir.path
+<!-- YAML
+added: REPLACEME
+-->
+
+* {string}
+
+The read-only path of this directory as was provided to [`fs.opendir()`][],
+[`fs.opendirSync()`][], or [`fsPromises.opendir()`][].
+
+### dir.close()
+<!-- YAML
+added: REPLACEME
+-->
+
+* Returns: {Promise}
+
+Asynchronously close the directory's underlying resource handle.
+Subsequent reads will result in errors.
+
+A `Promise` is returned that will be resolved after the resource has been
+closed.
+
+### dir.close(callback)
+<!-- YAML
+added: REPLACEME
+-->
+
+* `callback` {Function}
+  * `err` {Error}
+
+Asynchronously close the directory's underlying resource handle.
+Subsequent reads will result in errors.
+
+The `callback` will be called after the resource handle has been closed.
+
+### dir.closeSync()
+<!-- YAML
+added: REPLACEME
+-->
+
+Synchronously close the directory's underlying resource handle.
+Subsequent reads will result in errors.
+
+### dir.read([options])
+<!-- YAML
+added: REPLACEME
+-->
+
+* `options` {Object}
+  * `encoding` {string|null} **Default:** `'utf8'`
+* Returns: {Promise} containing {fs.Dirent}
+
+Asynchronously read the next directory entry via readdir(3) as an
+[`fs.Dirent`][].
+
+A `Promise` is returned that will be resolved with a [Dirent][] after the read
+is completed.
+
+_Directory entries returned by this function are in no particular order as
+provided by the operating system's underlying directory mechanisms._
+
+### dir.read([options, ]callback)
+<!-- YAML
+added: REPLACEME
+-->
+
+* `options` {Object}
+  * `encoding` {string|null} **Default:** `'utf8'`
+* `callback` {Function}
+  * `err` {Error}
+  * `dirent` {fs.Dirent}
+
+Asynchronously read the next directory entry via readdir(3) as an
+[`fs.Dirent`][].
+
+The `callback` will be called with a [Dirent][] after the read is completed.
+
+The `encoding` option sets the encoding of the `name` in the `dirent`.
+
+_Directory entries returned by this function are in no particular order as
+provided by the operating system's underlying directory mechanisms._
+
+### dir.readSync([options])
+<!-- YAML
+added: REPLACEME
+-->
+
+* `options` {Object}
+  * `encoding` {string|null} **Default:** `'utf8'`
+* Returns: {fs.Dirent}
+
+Synchronously read the next directory entry via readdir(3) as an
+[`fs.Dirent`][].
+
+The `encoding` option sets the encoding of the `name` in the `dirent`.
+
+_Directory entries returned by this function are in no particular order as
+provided by the operating system's underlying directory mechanisms._
+
+### dir\[Symbol.asyncIterator\]()
+<!-- YAML
+added: REPLACEME
+-->
+
+* Returns: {AsyncIterator} to fully iterate over all entries in the directory.
+
+_Directory entries returned by this iterator are in no particular order as
+provided by the operating system's underlying directory mechanisms._
+
 ## Class: fs.Dirent
 <!-- YAML
 added: v10.10.0
 -->
 
-When [`fs.readdir()`][] or [`fs.readdirSync()`][] is called with the
-`withFileTypes` option set to `true`, the resulting array is filled with
+A representation of a directory entry, as returned by reading from an [`fs.Dir`][].
+
+Additionally, when [`fs.readdir()`][] or [`fs.readdirSync()`][] is called with
+the `withFileTypes` option set to `true`, the resulting array is filled with
 `fs.Dirent` objects, rather than strings or `Buffers`.
 
 ### dirent.isBlockDevice()
@@ -2505,6 +2640,46 @@ Returns an integer representing the file descriptor.
 For detailed information, see the documentation of the asynchronous version of
 this API: [`fs.open()`][].
 
+## fs.opendir(path[, options], callback)
+<!-- YAML
+added: REPLACEME
+-->
+
+* `path` {string|Buffer|URL}
+* `options` {Object}
+  * `encoding` {string|null} **Default:** `'utf8'`
+* `callback` {Function}
+  * `err` {Error}
+  * `dir` {fs.Dir}
+
+Asynchronously open a directory. See opendir(3).
+
+Creates an [`fs.Dir`][], which contains all further functions for reading from
+and cleaning up the directory.
+
+The `encoding` option sets the encoding for the `path` while opening the
+directory and subsequent read operations (unless otherwise overriden during
+reads from the directory).
+
+## fs.opendirSync(path[, options])
+<!-- YAML
+added: REPLACEME
+-->
+
+* `path` {string|Buffer|URL}
+* `options` {Object}
+  * `encoding` {string|null} **Default:** `'utf8'`
+* Returns: {fs.Dir}
+
+Synchronously open a directory. See opendir(3).
+
+Creates an [`fs.Dir`][], which contains all further functions for reading from
+and cleaning up the directory.
+
+The `encoding` option sets the encoding for the `path` while opening the
+directory and subsequent read operations (unless otherwise overriden during
+reads from the directory).
+
 ## fs.read(fd, buffer, offset, length, position, callback)
 <!-- YAML
 added: v0.0.2
@@ -4644,6 +4819,39 @@ by [Naming Files, Paths, and Namespaces][]. Under NTFS, if the filename contains
 a colon, Node.js will open a file system stream, as described by
 [this MSDN page][MSDN-Using-Streams].
 
+## fsPromises.opendir(path[, options])
+<!-- YAML
+added: REPLACEME
+-->
+
+* `path` {string|Buffer|URL}
+* `options` {Object}
+  * `encoding` {string|null} **Default:** `'utf8'`
+* Returns: {Promise} containing {fs.Dir}
+
+Asynchronously open a directory. See opendir(3).
+
+Creates an [`fs.Dir`][], which contains all further functions for reading from
+and cleaning up the directory.
+
+The `encoding` option sets the encoding for the `path` while opening the
+directory and subsequent read operations (unless otherwise overriden during
+reads from the directory).
+
+Example using async interation:
+
+```js
+const fs = require('fs');
+
+async function print(path) {
+  const dir = await fs.promises.opendir(path);
+  for await (const dirent of dir) {
+    console.log(dirent.name);
+  }
+}
+print('./').catch(console.error);
+```
+
 ### fsPromises.readdir(path[, options])
 <!-- YAML
 added: v10.0.0
@@ -5253,6 +5461,7 @@ the file contents.
 [`UV_THREADPOOL_SIZE`]: cli.html#cli_uv_threadpool_size_size
 [`WriteStream`]: #fs_class_fs_writestream
 [`event ports`]: https://illumos.org/man/port_create
+[`fs.Dir`]: #fs_class_fs_dir
 [`fs.Dirent`]: #fs_class_fs_dirent
 [`fs.FSWatcher`]: #fs_class_fs_fswatcher
 [`fs.Stats`]: #fs_class_fs_stats
@@ -5269,6 +5478,8 @@ the file contents.
 [`fs.mkdir()`]: #fs_fs_mkdir_path_options_callback
 [`fs.mkdtemp()`]: #fs_fs_mkdtemp_prefix_options_callback
 [`fs.open()`]: #fs_fs_open_path_flags_mode_callback
+[`fs.opendir()`]: #fs_fs_opendir_path_options_callback
+[`fs.opendirSync()`]: #fs_fs_opendirsync_path_options
 [`fs.read()`]: #fs_fs_read_fd_buffer_offset_length_position_callback
 [`fs.readFile()`]: #fs_fs_readfile_path_options_callback
 [`fs.readFileSync()`]: #fs_fs_readfilesync_path_options
@@ -5284,6 +5495,7 @@ the file contents.
 [`fs.write(fd, string...)`]: #fs_fs_write_fd_string_position_encoding_callback
 [`fs.writeFile()`]: #fs_fs_writefile_file_data_options_callback
 [`fs.writev()`]: #fs_fs_writev_fd_buffers_position_callback
+[`fsPromises.opendir()`]: #fs_fspromises_opendir_path_options
 [`inotify(7)`]: http://man7.org/linux/man-pages/man7/inotify.7.html
 [`kqueue(2)`]: https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2
 [`net.Socket`]: net.html#net_class_net_socket