napi wasm bindings

[devsnek]

@josephg brought up a cool idea in #374 to expose napi to wasm. This would allow shipping abi stable binaries that are also platform independent(!!)

my initial exploration shows that at most our in-core changes are basically adding __attribute__((__import_module__("napi"))) to NAPI_EXTERN if __wasm32 is defined.

We could ship the bindings themselves as a separate module or include them in core, but they aren't actually too complex:

    napi_create_array: (envPtr, resultPtr) => {
      this.refreshMemory();
      this.writeValue(resultPtr, this.store([]));
      return NAPI_OK;
    },

At that point, you can do this:

[mhdawson]

@devsnek I was just talking to Lin Clark to set up a meeting between the N-API and WASM/WASI team to better understand. I was thinking that interacting between a WASM module and JavasScript Objects etc. was something where N-API might fit in and it looks like that you are thinking along the same lines.

What I want to figure out if this fits/makes sense with what the WASM team is planning.

Right now its looking like we'll hopefully have a meeting with the WASM team the first week of July to get the discussion going (they are very busy between now and then).

Do you have your experimental work somewhere I could clone and experiment with to learn more? I've not used WASM/WASI yet and this would be a good way for me to ramp up.

[devsnek]

i haven't uploaded napi anywhere yet but you can play with wasi by cloning this: nodejs/node#27850

[gabrielschulhof]

@devsnek I looked at the code in your PR and one thing I can't wrap my head around is how we would write a function that returns a JS value? All the examples AFAICT are self-contained programs that basically execute int main(void) {} from top to bottom.

N-API addons aren't really structured to be called from a main().

I guess I'd love to see an example of a simple library written as a WASM module.

[devsnek]

@gabrielschulhof

one thing I can't wrap my head around is how we would write a function that returns a JS value?

if you have this:

napi_value init(napi_env env, napi_value exports) {
  return exports;
}

it gets compiled to something like this in wasm:

(func $init (param $env i32) (param $exports i32) (result i32)
  (local.get $exports)
)

then from js you can simulate a handle scope with an array

const scope = [null, {}]; // 1 is exports
const out_idx = module.init(0, 1);
// `out_idx` is 1
const obj = scope[out_idx];

something like napi_create_object pushes a new value into the scope and returns the index.

you can see a mostly complete application of this concept here: https://gist.github.com/devsnek/db5499bf774f078e9ebb679680bd2cd1

N-API addons aren't really structured to be called from a main().

So what I did about this was:

#ifdef __wasm32__
#define NAPI_MODULE(modname, regfunc)                                 \
  NAPI_MODULE_EXPORT napi_value _napi_register(napi_env env,          \
                                               napi_value exports) {  \
    return regfunc(env, exports);                                     \
  }
#else
#define NAPI_MODULE(modname, regfunc)                                 \
  NAPI_MODULE_X(modname, regfunc, NULL, 0)  // NOLINT (readability/null_usage)
#endif

[OhadRau]

Hey just wanted to pipe in here since this is something I've been working on as well for the past few days. I'm approaching this a little differently, exposing N-API directly into WebAssembly using the Wasm C API (which just landed in V8's LKGR branch a few weeks ago). It's not immediately clear to me how all of N-API's functionality can be implemented in JS code alone, so I believe exposing N-API directly would get around potential issues in that regard. Also, performance-wise (after speaking with @ofrobots) it sounds like there is some edge to calling directly into C(++) rather than into JS. However some translation will have to happen between N-API and the WebAssembly wrapper library. I'm nearing a point where I'll be able to test some real libraries on this implementation, so hopefully I'll have a better picture of what the advantages/disadvantages of this approach are early next week.

(I spoke earlier today with @devsnek to get a picture of what he's working on and how we can work on this functionality without stepping on each other's feet too much. I've decided to continue working on my implementation so that we can better evaluate different approaches here and pick what makes the most sense for the Node community.)

[devsnek]

(fwiw i also feel that the capi implementation is the ideal way to go, i just was going for a quick mvp as we don't have a version of v8 with the wasm capi in node yet.)

[gabrielschulhof]

There may be another good reason to maintain a JS implementation of N-API, namely if we want to be able to run N-API addons in the browser. Unless V8 incorporates the N-API implementation we have in Node.js and makes it available as a WASM C API in the browser, we shall have to supply the JS implementation as a module that can be used to resolve a WASM-compiled N-API addon's napi_* symbols.

[OhadRau]

Yep @gabrielschulhof I definitely think it's worth pursuing both options here and evaluating them against each other. There's a very good chance that the JS version ends up working better for us here. For example, the overhead of doing everything in JS might be lower than the overhead of using the "actual" N-API to interface with JS (working on benchmarking this still).

Since some N-API functionality won't make sense in the browser (e.g. napi_get_uv_event_loop) I think it could be a good idea to define some subset of N-API that's safe to use in the browser/with WASM and then we can base the JS implementation off of that. (As of now I can't think of how we'd fit in the UV event loop to WASM modules anyways, so I think it makes sense to make it a native-only feature).

[gabrielschulhof]

@OhadRau done ✓🙂 https://github.com/nodejs/node/blob/d0e2650/src/js_native_api.h

[mhdawson]

As @gabrielschulhof mentioned about we've already separated out the Node and JS parts for this exact reason.

[mhdawson]

@OhadRau, @ofrobots any update on this?

[mhdawson]

From @ofrobots, unfortunately, they did not get as far along as they would have liked but these are the patches: https://github.com/OhadRau/node-v8/pulls

[mhdawson]

I think at this point we should close this issue and we can create a new one if there is a future effort. Please let us know if you think that was not the right thing to do.