You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'll explain. N-API brings to Neon the promise of a stable, backwards-compatible ABI—binary compatibility across all future versions of Node.
This is a big deal.
Portability across Node versions means Neon will finally be practical for publishing libraries, not just apps: a few prebuilt binaries should be sufficient for all downstream customers to use your native library without ever knowing the difference.
The stuff of legend, no?
Our Quest
Step 1. Create the feature flag
Create a cargo feature flag to allow us to concurrently maintain the main Neon codebase along with the experimental N-API support in the same master branch. (Merged!)
Set up a test suite specifically for the N-API backend so each task can easily include adding tests (Basic N-API tests #449)
Step 2. Implement the port
Module contexts and initialization: Implement the neon::context::ModuleContext type and pass it to the module initialization function inside the register_module! macro defined in /src/lib.rs. The context struct will likely need to encapsulate the underlying napi_env and napi_value as private fields. This can be implemented before we implement functions, with an unimplemented export_function() method for now.
Functions: This is probably one of the subtler tasks. See the implementation of neon::types::JsFunction::new(). The Rust callback can be stored as the extra void* data passed to napi_create_function.
Function arguments: Implement CallContext::len() and CallContext::argument().
Function returns: Implement function return values.
this: Implement CallContext::this().
Call kinds: Implement CallContext::kind().
Function exports: Once we have module contexts and functions implemented, we can implement the ModuleContext::export_function() shorthand method.
Objects: See neon::types::JsObject::new() and the neon::object::Object methods.
Arrays: See neon::types::JsArray.
ArrayBuffers and Buffers: See neon::types::binary and the N-API functions for working with binary data, such as napi_create_arraybuffer, napi_create_buffer, etc.
Uninitialized and null: These should be pretty bite-sized. See neon::types::JsUndefined and neon::types::JsNull. @goto-bus-stop
Strings: See neon::types::JsString. We'll need to explore what binary string representations can be used between the NAN vs N-API runtimes for constructing JS strings.
Classes: This will require us to figure out how to do unique branding with N-API, but I believe napi_define_class supports this. (Here is one pure C example we can look to for inspiration.) <== not needed for functional completeness; see Transition guide for porting to N-API backend #596
Errors: See neon::types::error. We'll need to explore how N-API does throwing and catching errors. - @anshulrgoyal 🔒
Conversions: See the uses of neon_runtime::convert::* and the napi_coerce_* functions.
Scopes: Luckily, the N-API HandleScope mechanim matches V8's mechanism very closely. See neon::context and the uses of various HandleScope internal types.
Tag checks: See uses of neon_runtime::tag::*.
Task scheduling: See neon::task and neon::context::TaskContext, and the N-API "simply asynchronous operations" API, which uses the same underlying libuv thread pool as Neon's current backend, but with N-API's stable ABI. <== not needed for functional completeness; see Transition guide for porting to N-API backend #596
Thread-safe callbacks: This can be implemented for N-API once we've merged an implementation for RFC 25, using napi_make_callback. <== not needed for functional completeness; see Transition guide for porting to N-API backend #596
Windows Support: Windows requires linking against node.lib and win_delay_load_hook. Create a custom build script to link these on windows.
We have just a couple remaining items to finish up:
Once we finish the complete port, we can switch the default feature flags to use the new runtime and publish a new 0.x minor version. Eventually after a few releases we can remove the old runtime completely.
How to Contribute
Building N-API-based projects
To experiment with the N-API runtime or do manual testing, you can create a Neon project that uses the right feature flags. To try it out, you can run:
neon new --no-default-features --features=napi-latest --neon=path/to/neon my-project
where path/to/neon is the path on your local filesystem to a local clone of the Neon repo.
Manual Steps
The output of neon new executed above will produce a project that fails to build. When using the neon backend, either neon-build should be used with a simple cargo build or neon-cli should be used and neon-build should be removed. If both are used, the project will fail to build.
There is an RFC (neon-bindings/rfcs#36) to replace neon new which will correctly generate a project. The simplest change is to edit native/Cargo.toml:
Remove the neon-build dependency
Remove build = "build.rs"
delete native/build.rs
Note: If you create a Neon project nested inside the directory tree of a clone of the Neon repo, you'll need to add the line
[workspace]
to your Neon project's native/Cargo.toml manifest in order to build the project.
Adding an N-API primitive
To add an N-API primitive, you should implement it in pure Rust (using unsafe as necessary, but only as necessary!) in crates/neon-runtime/napi, and call out to the N-API backend exposed through nodejs-sys.
When the Neon runtime needs to pass around a data structure, you can make two different definitions of the type, separated by testing the feature flag with #[cfg(feature = "...")]. You may sometimes need to refactor the types in the Neon runtime to accommodate differences between the legacy and N-API runtimes.
Adding a test
The test/napi directory is the space for adding N-API acceptance tests. You can add native Rust logic to test/napi/native/src and JS logic to test/napi/lib. You can get examples of existing acceptance tests in our existing backend in test/dynamic, which has the same structure.
Will You Join Us?
As you can see, the quest ahead of us will be no small feat!
Indeed, but fear not: we're here to help you if you get stuck. And many of these tasks can be a great way to get started with contributing to Neon and even learning Rust.
Claim one of the tasks today by leaving a comment below or pinging @dherman or @kjvalencik on Slack!
Prepare, once and future Neon contributors, for our noblest quest yet!
We are going to port Neon to Node's new N-API!
I'll explain. N-API brings to Neon the promise of a stable, backwards-compatible ABI—binary compatibility across all future versions of Node.
This is a big deal.
Portability across Node versions means Neon will finally be practical for publishing libraries, not just apps: a few prebuilt binaries should be sufficient for all downstream customers to use your native library without ever knowing the difference.
The stuff of legend, no?
Our Quest
Step 1. Create the feature flag
Step 2. Implement the port
neon::context::ModuleContexttype and pass it to the module initialization function inside theregister_module!macro defined in/src/lib.rs. The context struct will likely need to encapsulate the underlyingnapi_envandnapi_valueas private fields. This can be implemented before we implement functions, with an unimplementedexport_function()method for now.neon::types::JsFunction::new(). The Rust callback can be stored as the extravoid*data passed tonapi_create_function.CallContext::len()andCallContext::argument().this: ImplementCallContext::this().CallContext::kind().ModuleContext::export_function()shorthand method.neon::types::JsObject::new()and theneon::object::Objectmethods.neon::types::JsArray.neon::types::binaryand the N-API functions for working with binary data, such asnapi_create_arraybuffer,napi_create_buffer, etc.neon::types::JsUndefinedandneon::types::JsNull. @goto-bus-stopneon::types::JsBoolean. @goto-bus-stopneon::types::JsNumber.neon::types::JsString. We'll need to explore what binary string representations can be used between the NAN vs N-API runtimes for constructing JS strings.Classes: This will require us to figure out how to do unique branding with N-API, but I believe<== not needed for functional completeness; see Transition guide for porting to N-API backend #596napi_define_classsupports this. (Here is one pure C example we can look to for inspiration.)neon::types::error. We'll need to explore how N-API does throwing and catching errors. - @anshulrgoyal 🔒neon_runtime::convert::*and thenapi_coerce_*functions.neon::contextand the uses of various HandleScope internal types.neon_runtime::tag::*.Task scheduling: See<== not needed for functional completeness; see Transition guide for porting to N-API backend #596neon::taskandneon::context::TaskContext, and the N-API "simply asynchronous operations" API, which uses the same underlying libuv thread pool as Neon's current backend, but with N-API's stable ABI.Thread-safe callbacks: This can be implemented for N-API once we've merged an implementation for RFC 25, using<== not needed for functional completeness; see Transition guide for porting to N-API backend #596napi_make_callback.node.libandwin_delay_load_hook. Create a custom build script to link these on windows.We have just a couple remaining items to finish up:
JsBuffer::uninitialized- see JsBuffer::uninitialized() for N-API backend #664Step 3. Deprecate the legacy runtime
Once we finish the complete port, we can switch the default feature flags to use the new runtime and publish a new 0.x minor version. Eventually after a few releases we can remove the old runtime completely.
How to Contribute
Building N-API-based projects
To experiment with the N-API runtime or do manual testing, you can create a Neon project that uses the right feature flags. To try it out, you can run:
where
path/to/neonis the path on your local filesystem to a local clone of the Neon repo.Manual Steps
The output of
neon newexecuted above will produce a project that fails to build. When using theneonbackend, eitherneon-buildshould be used with a simplecargo buildorneon-clishould be used andneon-buildshould be removed. If both are used, the project will fail to build.There is an RFC (neon-bindings/rfcs#36) to replace
neon newwhich will correctly generate a project. The simplest change is to editnative/Cargo.toml:neon-builddependencybuild = "build.rs"native/build.rsNote: If you create a Neon project nested inside the directory tree of a clone of the Neon repo, you'll need to add the line
to your Neon project's
native/Cargo.tomlmanifest in order to build the project.Adding an N-API primitive
To add an N-API primitive, you should implement it in pure Rust (using
unsafeas necessary, but only as necessary!) incrates/neon-runtime/napi, and call out to the N-API backend exposed throughnodejs-sys.When the Neon runtime needs to pass around a data structure, you can make two different definitions of the type, separated by testing the feature flag with
#[cfg(feature = "...")]. You may sometimes need to refactor the types in the Neon runtime to accommodate differences between the legacy and N-API runtimes.Adding a test
The
test/napidirectory is the space for adding N-API acceptance tests. You can add native Rust logic totest/napi/native/srcand JS logic totest/napi/lib. You can get examples of existing acceptance tests in our existing backend intest/dynamic, which has the same structure.Will You Join Us?
As you can see, the quest ahead of us will be no small feat!
Indeed, but fear not: we're here to help you if you get stuck. And many of these tasks can be a great way to get started with contributing to Neon and even learning Rust.
Claim one of the tasks today by leaving a comment below or pinging @dherman or @kjvalencik on Slack!