summaryrefslogtreecommitdiff
path: root/tools/test.py
AgeCommit message (Collapse)Author
2019-08-10test: use unique tmpdirs for each testJoão Reis
Tests can leave processes running blocking the tmpdir. This does not yet prevent tests from doing that, but prevents failures on subsequent tests. PR-URL: https://github.com/nodejs/node/pull/28858 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2019-07-15test: fix pty test hangs on aixBen Noordhuis
Some pty tests persistently hung on the AIX CI buildbots. Fix that by adding a helper script that properly sets up the pty before spawning the script under test. On investigation I discovered that the test runner hung when it tried to close the slave pty's file descriptor, probably due to a bug in AIX's pty implementation. I could reproduce it with a short C program. The test runner also leaked file descriptors to the child process. I couldn't convince python's `subprocess.Popen()` to do what I wanted it to do so I opted to move the logic to a helper script that can do fork/setsid/etc. without having to worry about stomping on state in tools/test.py. In the process I also uncovered some bugs in the pty module of the python distro that ships with macOS 10.14, leading me to reimplement a sizable chunk of the functionality of that module. And last but not least, of course there are differences between ptys on different platforms and the helper script has to paper over that. Of course. Really, this commit took me longer to put together than I care to admit. Caveat emptor: this commit takes the hacky ^D feeding to the slave out of tools/test.py and puts it in the *.in input files. You can also feed other control characters to tests, like ^C or ^Z, simply by inserting them into the corresponding input file. I think that's nice. Fixes: https://github.com/nodejs/build/issues/1820 Fixes: https://github.com/nodejs/node/issues/28489 PR-URL: https://github.com/nodejs/node/pull/28600 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>
2019-04-13tools: python: ignore instead of select flake8 rulesRefael Ackermann
PR-URL: https://github.com/nodejs/node/pull/25614 Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2019-04-13tools: python: activate more flake8 rulesRefael Ackermann
PR-URL: https://github.com/nodejs/node/pull/25614 Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2019-04-02tools: fix `test.py --time`Richard Lau
PR-URL: https://github.com/nodejs/node/pull/27007 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2019-03-08tools: fix test.py --shellYang Guo
This fixes the necessary plumbing to make the --shell argument have its intended effect. PR-URL: https://github.com/nodejs/node/pull/26449 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2019-02-17test: add --test-root option to test.pyYang Guo
This way we can specify a custom path for the test folder, e.g. when building addons separately from the source tree. PR-URL: https://github.com/nodejs/node/pull/26093 Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com>
2019-02-04test: do not fail SLOW tests if they are not slowYang Guo
PR-URL: https://github.com/nodejs/node/pull/25868 Refs: https://github.com/nodejs/node/issues/25867 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Gus Caplan <me@gus.host>
2019-02-02tools: make test.py Python 3 compatibleSakthipriyan Vairamani (thefourtheye)
PR-URL: https://github.com/nodejs/node/pull/25767 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: cclauss <cclauss@me.com> Reviewed-By: gengjiawen <technicalcute@gmail.com>
2019-02-02process: expose process.features.inspectorJoyee Cheung
Instead of using process.config.variables.v8_enable_inspector to detect whether inspector is enabled in the build. PR-URL: https://github.com/nodejs/node/pull/25819 Refs: https://github.com/nodejs/node/issues/25343 Reviewed-By: Eugene Ostroukhov <eostroukhov@google.com> Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>
2019-01-29test: enable marking of failing coverage testsMichael Dawson
Enable marking of coverage tests so that we can allow some tests to fail without blocking the generation of coverage data. This will later allow us to fail the coverage job if other kinds of errors occur and to capture which tests we believe are not running properly with coverage enabled. PR-URL: https://github.com/nodejs/node/pull/25671 Reviewed-By: Ben Coe <bencoe@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Refael Ackermann <refack@gmail.com>
2019-01-29tools: make test.py Queue part Python 3 compatiblegengjiawen
Signed-off-by: gengjiawen <technicalcute@gmail.com> PR-URL: https://github.com/nodejs/node/pull/25701 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2019-01-09worker: remove `--experimental-worker` flagAnna Henningsen
Having an experimental feature behind a flag makes change if we are expecting significant breaking changes to its API. Since the Worker API has been essentially stable since its initial introduction, and no noticeable doubt about possibly not keeping the feature around has been voiced, removing the flag and thereby reducing the barrier to experimentation, and consequently receiving feedback on the implementation, seems like a good idea. PR-URL: https://github.com/nodejs/node/pull/25361 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Yuta Hiroto <hello@hiroppy.me> Reviewed-By: Shingo Inoue <leko.noor@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Masashi Hirano <shisama07@gmail.com> Reviewed-By: Weijia Wang <starkwang@126.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
2019-01-02test: set umask explicitlyThomas Chung
Some tests which create files and check file permissions assume the umask is compatible with 022, and break when set to something like 007. Explicitly set umask to 022 PR-URL: https://github.com/nodejs/node/pull/25213 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2018-12-24tools: alphabetize IGNORED_SUITES in tools/test.pyRich Trott
PR-URL: https://github.com/nodejs/node/pull/25182 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2018-12-10tools: prepare tools/testp.py for Python 3cclauss
PR-URL: https://github.com/nodejs/node/pull/24890 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
2018-12-07tools: prepare tools/test.py for Python 3cclauss
PR-URL: https://github.com/nodejs/node/pull/24799 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
2018-12-04test: partition N-API testsGabriel Schulhof
Partition test/addons-napi into test/js-native-api and test/node-api to isolate the Node.js-agnostic portion of the N-API tests from the Node.js-specific portion. PR-URL: https://github.com/nodejs/node/pull/24557 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
2018-12-03build: add line break as soon tests are doneRuben Bridgewater
PR-URL: https://github.com/nodejs/node/pull/24748 Refs: https://github.com/nodejs/node/pull/24486 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Michaël Zasso <targos@protonmail.com>
2018-12-03build: fix line length off by one errorRuben Bridgewater
While running the test suite the progress bar shows former line endings if the new line is shorter than the former line. The length was calculated without the line ending. It is now an empty string to prevent the off by one error instead of using extra whitespace. PR-URL: https://github.com/nodejs/node/pull/24748 Refs: https://github.com/nodejs/node/pull/24486 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Michaël Zasso <targos@protonmail.com>
2018-11-25tools: use print() function on both Python 2 and 3cclauss
PR-URL: https://github.com/nodejs/node/pull/24486 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2018-11-18test: skip test that use --tls-v1.x flagsDaniel Bevenius
Currently, configuring --without-ssl will cause the following test to fail: === release test-https-agent-additional-options === Path: parallel/test-https-agent-additional-options out/Release/node: bad option: --tls-v1.1 Command: out/Release/node --tls-v1.1 /node/test/parallel/test-https-agent-additional-options.js === release test-https-agent-session-eviction === Path: parallel/test-https-agent-session-eviction out/Release/node: bad option: --tls-v1.0 Command: out/Release/node --tls-v1.0 /node/test/parallel/test-https-agent-session-eviction.js This commit adds a check for the --tls-v.x flags and skips them if node was built without crypto support. PR-URL: https://github.com/nodejs/node/pull/24376 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com> Reviewed-By: Rod Vagg <rod@vagg.org>
2018-11-10test: move benchmark tests out of main test suiteRich Trott
Move benchmark tests (which are slow) out of the main test suite. We can hopefully add them to node-daily-master so that they are still run daily on CI. PR-URL: https://github.com/nodejs/node/pull/24265 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2018-10-30test: run code cache test by default and test generatorJoyee Cheung
- Add the code cache tests to the default test suite, and test the bookkeeping when the binary is not built with the code cache. - Test the code cache generator to make sure we do not accidentally break it - until we enable code cache in the CI. Refs: https://github.com/nodejs/node/issues/21563 PR-URL: https://github.com/nodejs/node/pull/23855 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2018-10-18tools: prefer filter to remove empty stringsSakthipriyan Vairamani (thefourtheye)
Ref: https://github.com/nodejs/node/pull/23585#issuecomment-430585490 Python's `list.remove` will throw if the element is not found and also it removes only the first occurrence. This patch replaces the use of `list.remove` with a `filter` which solves both of the above mentioned problems. PR-URL: https://github.com/nodejs/node/pull/23727 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Matheus Marchini <mat@mmarchini.me> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2018-10-16tools,test: add list of slow testsRefael Ackermann
they will be skipped if run with `--flaky-tests=skip` PR-URL: https://github.com/nodejs/node/pull/23251 Reviewed-By: Rich Trott <rtrott@gmail.com>
2018-10-16tools,test: cleanup and dedup codeRefael Ackermann
* Hoist common code to base class (`GetTestStatus`, and the `section` property to `TestConfiguration`) * Replace ListSet with the built in set * Remove ClassifiedTest * Inline PrintReport * How cases_to_run are filtered PR-URL: https://github.com/nodejs/node/pull/23251 Reviewed-By: Rich Trott <rtrott@gmail.com>
2018-10-04tools: allow input for TTY testsAnna Henningsen
Since faking TTY input is not otherwise fake-able, we need support in the test runner for it. PR-URL: https://github.com/nodejs/node/pull/23053 Reviewed-By: James M Snell <jasnell@gmail.com>
2018-08-01test: remove test/gc, integrate into parallelAnna Henningsen
There’s no reason to have a separate addon just for testing GC anymore. PR-URL: https://github.com/nodejs/node/pull/22001 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Jon Moss <me@jonathanmoss.me> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2018-07-31tools,test: remove unused config hooksJon Moss
Removes a couple of unused/empty functions inside of `tools/test.py` PR-URL: https://github.com/nodejs/node/pull/22010 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com>
2018-07-26tools: fix docs and run known_issues by defaultJon Moss
- Updates `test/README.md` with new suites - Fixes some outdated `IGNORED_SUITES` listings - Allows for `test/known_issues` suite to be run by default PR-URL: https://github.com/nodejs/node/pull/21910 Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Lance Ball <lball@redhat.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Rich Trott <rtrott@gmail.com>
2018-06-27build: speed up startup with V8 code cacheJoyee Cheung
This patch speeds up the startup time and reduce the startup memory footprint by using V8 code cache when comiling builtin modules. The current approach is demonstrated in the `with-code-cache` Makefile target (no corresponding Windows target at the moment). 1. Build the binary normally (`src/node_code_cache_stub.cc` is used), by now `internalBinding('code_cache')` is an empty object 2. Run `tools/generate_code_cache.js` with the binary, which generates the code caches by reading source code of builtin modules off source code exposed by `require('internal/bootstrap/cache').builtinSource` and then generate a C++ file containing static char arrays of the code cache, using a format similar to `node_javascript.cc` 3. Run `configure` with the `--code-cache-path` option so that the newly generated C++ file will be used when compiling the new binary. The generated C++ file will put the cache into the `internalBinding('code_cache')` object with the module ids as keys 4. The new binary tries to read the code cache from `internalBinding('code_cache')` and use it to compile builtin modules. If the cache is used, it will put the id into `require('internal/bootstrap/cache').compiledWithCache` for bookkeeping, otherwise the id will be pushed into `require('internal/bootstrap/cache').compiledWithoutCache` This patch also added tests that verify the code cache is generated and used when compiling builtin modules. The binary with code cache: - Is ~1MB bigger than the binary without code cahe - Consumes ~1MB less memory during start up - Starts up about 60% faster PR-URL: https://github.com/nodejs/node/pull/21405 Reviewed-By: John-David Dalton <john.david.dalton@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Gus Caplan <me@gus.host>
2018-06-06test: create new directory v8-updatesMatheus Marchini
`v8-updates` holds all tests related to V8 updates, for example, testing for postmortem metadata chages. PR-URL: https://github.com/nodejs/node/pull/20783 Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-06-06test,tools: enable running tests under workersAnna Henningsen
Enable running tests inside workers by passing `--worker` to `tools/test.py`. A number of tests are marked as skipped, or have been slightly altered to fit the different environment. PR-URL: https://github.com/nodejs/node/pull/20876 Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Shingo Inoue <leko.noor@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: John-David Dalton <john.david.dalton@gmail.com> Reviewed-By: Gus Caplan <me@gus.host>
2018-05-22test: fix tests that fail under coverageBenjamin Coe
Make test runner capable of skipping tests, which makes it possible to skip the failing test/message/core_line_numbers.js test. Make nyc no longer generate compact instrumentation (this causes significantly different code output, which leads to failing test assertions). PR-URL: https://github.com/nodejs/node/pull/20794 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Khaidi Chu <i@2333.moe> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
2018-05-02tools: fix TypeError from `test.py --time`Richard Lau
Calculated durations are timedelta objects but the FormatTime function is expecting a number in seconds. PR-URL: https://github.com/nodejs/node/pull/20368 Fixes: https://github.com/nodejs/node/issues/20341 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2018-04-27tools: add log output to crashesRuben Bridgewater
PR-URL: https://github.com/nodejs/node/pull/20295 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Matheus Marchini <matheus@sthima.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2018-04-26tools: show stdout/stderr for timed out testsRich Trott
When a test times out, the contents of stdout and stderr can often be highly valuable in debugging. Provide that information. Refs: https://github.com/nodejs/node/issues/19906#issuecomment-382073888 PR-URL: https://github.com/nodejs/node/pull/20260 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com> Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2018-04-23test: fix test when NODE_OPTIONS env var is set to --trace-warningsAshok
PR-URL: https://github.com/nodejs/node/pull/20027 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2018-04-13tools: treat SIGABRT as crashAnna Henningsen
Otherwise, potentially no output is shown for aborts. PR-URL: https://github.com/nodejs/node/pull/19990 Refs: https://github.com/nodejs/node/issues/19903 Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>
2018-04-11tools: include exit code in TAP logRefael Ackermann
PR-URL: https://github.com/nodejs/node/pull/19855 Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Rod Vagg <rod@vagg.org>
2018-04-11tools: include exit code in test failuresRich Trott
Include the exit code in test failures. This will give us more information during the currently-puzzling failures that provide no information in CI such as: ``` 03:10:10 not ok 563 parallel/test-fs-truncate 03:10:10 --- 03:10:10 duration_ms: 1.119 03:10:10 severity: fail 03:10:10 stack: |- 03:10:10 ... ``` PR-URL: https://github.com/nodejs/node/pull/19855 Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Rod Vagg <rod@vagg.org>
2017-12-11tools,test: throw if common.PORT used in parallel testsRich Trott
common.PORT should not be used in parallelized tests. (There can be a port collision if another tests requests an arbitrary open port from the operating system and ends up getting common.PORT before a test that uses common.PORT uses the port.) In such a situation, throw an error. PR-URL: https://github.com/nodejs/node/pull/17559 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
2017-11-29tools,test: use Execute instead of check_outputRefael Ackermann
subprocess.check_output is a python2.7 only feature. Using Execute allows keeping python2.6 compatibility PR-URL: https://github.com/nodejs/node/pull/17381 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Myles Borins <myles.borins@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
2017-11-28test: Enable specifying flaky tests on fipsNikhil Komawar
Adds a way to mark a specified test as 'flaky' on fips compliant systems. Earlier, the ``tools/test.py`` script supported only 'mode', 'system' and 'arch' for test environment specification. This limits the ability to specify the behavior of tests and setting pre-determined behavior of the same on other types of systems. As an example, the feature request below indicates the need to specify certain tests as 'flaky' on fips compliant systems. It hints at future possibility of a shared library, which in turn may need a specifier for running tests. This commit introduces a new item in the ``env`` dict, called ``type`` which defaults to ``simple`` type. It also adds an optional command line argument ``--type``, which inputs strings. Current functionality extends to setting ``simple`` or ``fips`` for this ``type`` variable. However, extending it to further uses is rather simple by adding "if" conditions at appropriate places in the ``tools/test.py`` script. PR-URL: https://github.com/nodejs/node/pull/16329 Fixes: https://github.com/nodejs/node/issues/14746 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2017-11-20tools: allow running test.py without configuringGibson Fahnestock
If config.gypi isn't defined, assume Node was build the default way, i.e. with the inspector. PR-URL: https://github.com/nodejs/node/pull/16621 Refs: https://github.com/nodejs/node/pull/16436#issuecomment-340115175 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
2017-11-17tools: fail tests if malformed status fileRich Trott
PR-URL: https://github.com/nodejs/node/pull/16703 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2017-10-19test: fix flaky test-make-docRich Trott
`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js` runs the test but does not build the docs. We do not want to build the docs in these cases. Move the test to the `doctool` suite and add that suite to `IGNORED_SUITES` in `test.py`. Specify `doctool` as a test suite to run with `make test` or `make test-ci` but not with the `make test-ci-js` job run on cross-compiled fanned CI tests like we do with Raspberry Pi devices in CI. PR-URL: https://github.com/nodejs/node/pull/16301 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2017-10-19tools: rename unused variale in more pythonic wayNikhil Komawar
The 'Main' function in tools/test.py file was using a variable named ``all_outcomes`` to store a value not being used. It is a best practice to name unused variables, often return values of functions/methods (as in this case) as ``_`` [1]. This just helps keep the code a bit cleaner and avoid any silly mistakes. PR-URL: https://github.com/nodejs/node/pull/16171 Refs: [1] https://stackoverflow.com/a/5477153 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Claudio Rodriguez <cjrodr@yahoo.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2017-10-01test: make it easier to run tests for subsystemsBenjamin Coe
You can now run suites for subsystem using shorthand, e.g., http. Switch to black-list of default test folders from white-list. Tests run by 'make test', 'make coverage', etc., now configurable. Stop running known_issues suite when collecting test coverage. PR-URL: https://github.com/nodejs/node/pull/15450 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-05-11 14:08:24 +0000