summaryrefslogtreecommitdiff
path: root/tools/js2c.py
AgeCommit message (Collapse)Author
2019-08-12tools: allow single JS file for --link-moduleDaniel Bevenius
The description for the --link-module configuration option is as follows: $ ./configure --help | grep -A 5 'link-module' --link-module=LINKED_MODULE Path to a JS file to be bundled in the binary as a builtin. This module will be referenced by path without extension; e.g. /root/x/y.js will be referenced via require('root/x/y'). Can be used multiple times This lead me to think that it was possible to specify a file like this: $ ./configure --link-module=something.js $ NODE_DEBUG=mkcodecache make -j8 This will lead to a compilation error as an entry in the source_ map in node_javascript.cc will end up having an empty string as its key: source_.emplace("", UnionBytes{_raw, 105}); This will then be used by CodeCacheBuilder when it iterates over the module ids, which will lead to the following compilation errors: /node/out/Release/obj/gen/node_code_cache.cc:12:23: warning: ISO C++17 does not allow a decomposition group to be empty [-Wempty-decomposition] static const uint8_t [] = { ^ /node/out/Release/obj/gen/node_code_cache.cc:12:22: warning: decomposition declarations are a C++17 extension [-Wc++17-extensions] static const uint8_t [] = { ^~ /node/out/Release/obj/gen/node_code_cache.cc:12:1: error: decomposition declaration cannot be declared 'static' static const uint8_t [] = { ^~~~~~ /node/out/Release/obj/gen/node_code_cache.cc:12:22: error: decomposition declaration cannot be declared with type 'const uint8_t' (aka 'const unsigned char'); declared type must be 'auto' or reference to 'auto' static const uint8_t [] = { ^ /node/out/Release/obj/gen/node_code_cache.cc:12:22: error: excess elements in scalar initializer static const uint8_t [] = { ^ /node/out/Release/obj/gen/node_code_cache.cc:660:7: error: expected expression , ^ /node/out/Release/obj/gen/node_code_cache.cc:661:24: error: no matching function for call to 'arraysize' static_cast<int>(arraysize()), policy ^~~~~~~~~ ../src/util.h:667:18: note: candidate function template not viable: requires 1 argument, but 0 were provided constexpr size_t arraysize(const T (&)[N]) { ^ 2 warnings and 5 errors generated. This commit suggests that passing a single file be allowed by modifying tools/js2c.py. PR-URL: https://github.com/nodejs/node/pull/28443 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>
2019-06-28tools: fix typo in js2c.pyDaniel Bevenius
PR-URL: https://github.com/nodejs/node/pull/28417 Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2019-06-02tools: fix js2c regressionRefael Ackermann
PR-URL: https://github.com/nodejs/node/pull/27980 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2019-05-23src: remove env-inl.h from header filesSam Roberts
Inline headers should only be included into the .cc files that use them. PR-URL: https://github.com/nodejs/node/pull/27755 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com>
2019-05-18tools: refactor js2c.py for maximal Python3 compatibilityRefael Ackermann
* add explicit `--target` argument to enable succinct gyp declaration * simplify js2c semantics PR-URL: https://github.com/nodejs/node/pull/25518 Reviewed-By: Christian Clauss <cclauss@me.com>
2019-04-13tools: python: ignore instead of select flake8 rulesRefael Ackermann
PR-URL: https://github.com/nodejs/node/pull/25614 Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2019-04-13tools: python: activate more flake8 rulesRefael Ackermann
PR-URL: https://github.com/nodejs/node/pull/25614 Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2018-12-30lib: remove unintended access to deps/Anna Henningsen
This brings DEP0084 to End-of-Life. It is unlikely that this has received much public usage in the first place, so removing should be okay. PR-URL: https://github.com/nodejs/node/pull/25138 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2018-12-18src: remove code cache integrity checkJoyee Cheung
In preparation of sharing code cache among different threads - we simply rely on v8 to reject invalid cache, since there isn't any serious consequence when the cache is invalid anyway. PR-URL: https://github.com/nodejs/node/pull/24950 Reviewed-By: Anna Henningsen <anna@addaleax.net>
2018-12-11process: specialize building and storage of process.configJoyee Cheung
Instead of treating config.gypi as a JavaScript file, specialize the processing in js2c and make the serialized result a real JSON string (with 'true' and 'false' converted to boolean values) so we don't have to use a custom deserializer during bootstrap. In addition, store the JSON string separately in NativeModuleLoader, and keep it separate from the map of the builtin source code, so we don't have to put it onto `NativeModule._source` and delete it later, though we still preserve it in `process.binding('natives')`, which we don't use anymore. This patch also makes the map of builtin source code and the config.gypi string available through side-effect-free getters in C++. PR-URL: https://github.com/nodejs/node/pull/24816 Reviewed-By: Gus Caplan <me@gus.host>
2018-12-07tools: prepare tools/js2c.py for Python 3cclauss
PR-URL: https://github.com/nodejs/node/pull/24798 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>
2018-11-20src: use STL containers instead of v8 values for static module dataJoyee Cheung
Instead of putting the source code and the cache in v8::Objects, put them in per-process std::maps. This has the following benefits: - It's slightly lighter in weight compared to storing things on the v8 heap. Also it may be slightly faster since the preivous v8::Object is already in dictionary mode - though the difference is very small given the number of native modules is limited. - The source and code cache generation templates are now much simpler since they just initialize static arrays and manipulate STL constructs. - The static native module data can be accessed independently of any Environment or Isolate, and it's easy to look them up from the C++'s side. - It's now impossible to mutate the source code used to compile native modules from the JS land since it's completely separate from the v8 heap. We can still get the constant strings from process.binding('natives') but that's all. A few drive-by fixes: - Remove DecorateErrorStack in LookupAndCompile - We don't need to capture the exception to decorate when we encounter errors during native module compilation, as those errors should be syntax errors and v8 is able to decorate them well. We use CompileFunctionInContext so there is no need to worry about wrappers either. - The code cache could be rejected when node is started with v8 flags. Instead of aborting in that case, simply keep a record in the native_module_without_cache set. - Refactor js2c.py a bit, reduce code duplication and inline Render() to make the one-byte/two-byte special treatment easier to read. PR-URL: https://github.com/nodejs/node/pull/24384 Fixes: https://github.com/Remove Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com> Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
2018-08-18src: perform integrity checks on built-in code cacheJoyee Cheung
Currently V8 only checks that the length of the source code is the same as the code used to generate the hash, so we add an additional check here: 1. During compile time, when generating node_javascript.cc and node_code_cache.cc, we compute and include the hash of the (unwrapped) JavaScript source in both. 2. At runtime, we check that the hash of the code being compiled and the hash of the code used to generate the cache (inside the wrapper) is the same. This is based on the assumptions: 1. `internalBinding('code_cache_hash')` must be in sync with `internalBinding('code_cache')` (same C++ file) 2. `internalBinding('natives_hash')` must be in sync with `process.binding('natives')` (same C++ file) 3. If `internalBinding('natives_hash')` is in sync with `internalBinding('natives_hash')`, then the (unwrapped) code used to generate `internalBinding('code_cache')` should be in sync with the (unwrapped) code in `process.binding('natives')` There will be, however, false positives if the wrapper used to generate the cache is different from the one used at run time, and the length of the wrapper somehow stays the same. But that should be rare and can be eased once we make the two bootstrappers cached and checked as well. PR-URL: https://github.com/nodejs/node/pull/22152 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Gus Caplan <me@gus.host>
2018-06-27src: move context bootstrap to jsGus Caplan
PR-URL: https://github.com/nodejs/node/pull/21518 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Anna Henningsen <anna@addaleax.net>
2018-06-11tools: add option to use custom template with js2c.pyShelley Vohr
PR-URL: https://github.com/nodejs/node/pull/21187 Reviewed-By: Anna Henningsen <anna@addaleax.net>
2018-05-14tools: hide symbols for builtin JS files in binaryAnna Henningsen
Do not expose symbols like `node::internal_process_next_tick_value`, `node::internal_process_next_tick_key` in the created `node` binary by wrapping them in an anonymous namespace. PR-URL: https://github.com/nodejs/node/pull/20634 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
2018-03-09tools,bootstrap: preprocess gypi files to jsonGus Caplan
PR-URL: https://github.com/nodejs/node/pull/19140 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2018-03-06src: move internal loaders out of bootstrap_node.jsJoyee Cheung
- Moves the creation of `process.binding()`, `process._linkedBinding()` `internalBinding()` and `NativeModule` into a separate file `lib/internal/bootstrap_loaders.js`, and documents them there. This file will be compiled and run before `bootstrap_node.js`, which means we now bootstrap the internal module & binding system before actually bootstrapping Node.js. - Rename the special ID that can be used to require `NativeModule` as `internal/bootstrap_loaders` since it is setup there. Also put `internalBinding` in the object exported by `NativeModule.require` instead of putting it inside the `NativeModule.wrapper` - Use the original `getBinding()` to get the source code of native modules instead of getting it from `process.binding('native')` so that users cannot fake native modules by modifying the binding object. - Names the bootstrapping functions so their names show up in the stack trace. PR-URL: https://github.com/nodejs/node/pull/19112 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Gus Caplan <me@gus.host>
2018-03-05lib: add internal check macrosGus Caplan
PR-URL: https://github.com/nodejs/node/pull/18852 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2017-11-13build: runtime-deprecate requiring depsTimothy Gu
PR-URL: https://github.com/nodejs/node/pull/16392 Fixes: https://github.com/nodejs/node/pull/15566#discussion_r146111170 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2017-02-13src: support UTF-8 in compiled-in JS source filesBen Noordhuis
Detect it when source files in lib/ are not ASCII. Decode them as UTF-8 and store them as UTF-16 in the binary so they can be used as external string resources without non-ASCII characters getting mangled. Fixes: https://github.com/nodejs/node/issues/10673 PR-URL: https://github.com/nodejs/node/pull/11129 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com>
2016-10-25src,tools: speed up startup by 2.5%Ben Noordhuis
Use zero-copy external string resources for storing the built-in JS source code. Saves a few hundred kilobyte of memory and consistently speeds up `benchmark/misc/startup.js` by 2.5%. Everything old is new again! Commit 74954ce ("Add string class that uses ExternalAsciiStringResource.") from 2011 did the same thing but I removed that in 2013 in commit 34b0a36 ("src: don't use NewExternal() with unaligned strings") because of a limitation in the V8 API. V8 no longer requires that strings are aligned if they are one-byte strings so it should be safe to re-enable external strings again. PR-URL: https://github.com/nodejs/node/pull/5458 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Robert Jefe Lindstaedt <robert.lindstaedt@gmail.com>
2016-05-12tools: remove the minifying logicSakthipriyan Vairamani
As the minifier logic is not used at all, this patch removes the code necessary for it. PR-URL: https://github.com/nodejs/node/pull/6636 Reviewed-By: Jackson Tian <shvyo1987@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2016-02-25src,tools: remove null sentinel from source arrayBen Noordhuis
PR-URL: https://github.com/nodejs/node/pull/5418 Reviewed-By: Trevor Norris <trev.norris@gmail.com>
2016-02-25src,tools: drop nul byte from built-in source codeBen Noordhuis
PR-URL: https://github.com/nodejs/node/pull/5418 Reviewed-By: Trevor Norris <trev.norris@gmail.com>
2016-02-25src,tools: allow utf-8 in built-in js source codeBen Noordhuis
PR-URL: https://github.com/nodejs/node/pull/5418 Reviewed-By: Trevor Norris <trev.norris@gmail.com>
2015-12-08tools: add --prof-process flag to node binaryMatt Loring
This change cleans up outstanding comments on #3032. It improves error handling when no isolate file is provided and adds the --prof-process flag to the node binary which executes the tick processor on the provided isolate file. PR-URL: https://github.com/nodejs/node/pull/4021 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Evan Lucas <evanlucas@me.com>
2015-09-15build: Updates to enable AIX supportMichael Dawson
These are the core changes that allow AIX to compile. There are still some test failures as there are some patches needed for libuv and npm that we'll need to contribute through those communities but this set allows node to be built on AIX and pass most of the core tests The change in js2c is because AIX does not support $ in identifier names. See the discussion/agreement in https://github.com/nodejs/node/issues/2272 PR-URL: https://github.com/nodejs/node/pull/2364 Reviewed-By: Ben Noordhuis <ben@strongloop.com> Reviewed-By: Rod Vagg <r@va.gg>
2015-03-27js2c: fix module id generation on windowsBen Noordhuis
Fix a regression that was introduced in commit 2db758c ("iojs: introduce internal modules") where the computed id for "config.gypi" on Windows was not "config" but an empty string. With an empty string, the build succeeds but the binary is unusable: startup.processConfig() in src/node.js chokes on the missing .config property. PR-URL: https://github.com/iojs/io.js/pull/1281 Reviewed-By: Vladimir Kurchatkin <vladimir.kurchatkin@gmail.com>
2015-03-25iojs: introduce internal modulesVladimir Kurchatkin
Internal modules can be used to share private code between public modules without risk to expose private APIs to the user. PR-URL: https://github.com/iojs/io.js/pull/848 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2013-07-24tools: fix js2c macro expansion bugBen Noordhuis
If the same macro was used twice in close proximity, the second one didn't get expanded.
2012-11-24js2c: raise proper Exception, not a stringBen Noordhuis
Fixes the following error message: TypeError: exceptions must be old-style classes or derived from BaseException, not str Fixes #4303.
2012-11-21windows: add tracing with performance countersScott Blomquist
Patch by Henry Rawas and Scott Blomquist.
2012-08-24tools: fix missing initializer warning in js2c.pyBen Noordhuis
Fix a -Wmissing-field-initializers style compiler warning in the code that's generated by js2c.py.
2012-03-15js2c: fix to support files other than ones ending with 2 char extensionsNathan Rajlich
Previously this was basically hard-coded for *.js files, but now we need to include the 'config.gypi' file in there as well.
2011-03-08Add string class that uses ExternalAsciiStringResource.Tom Hughes
Change the natives to use this class instead of creating completely new strings. Reduces memory usage by about 1 MB.
2010-12-20Better way of getting rid of symlinksBert Belder
Windows, especially msysgit, doesn't like 'em
2010-11-22Add cmake build support.Tom Hughes
Squashed commit of ca128f7dcd28cbcfba154c8577ed54d4aa71dd02 with contributions from Mark Constable (markc@renta.net) and Daniel Gröber (darklord@darkboxed.org).
2010-11-01Back to WAFRyan Dahl
Too much instability right now. Hopefully we can revisit autoconf soon.
2010-10-24Replace WAF with make/autoconfRyan Dahl
2010-10-12Dynamically load native scriptsRyan Dahl
2010-03-15Include lib/ directory in node executable. Compile on demand.Ryan Dahl
Instead of installing the files in /usr/lib/node/libraries and loading them from the file system, the files are built-in to the node executable. However, they are only compiled on demand. The reasoning is: 1. Allow for more complex internal javascript. In particular, process.stdout and process.stdin can be js implemented streams. 2. Ease system installs. Loading from disk each time is unnecessary overhead. Note that there is no "system" path for modules anymore. Only $HOME/.node_libraries.
2010-02-08Use Google's jsmin.py so that Node can be used for evilRyan Dahl
2009-10-07Fix build.Ryan Dahl
- Add -f flag to rm on make clean. - Use old jsmin - the new one seems to be broken.
2009-10-07Upgrade v8 to 1.3.14Ryan Dahl
2009-06-22Create tools directoryRyan
Users will need to "make distclean" to recompile from scratch since I've moved a lot of the core build tools around.