WASI threads support

[loganek]

I create this issue to track implementation of the WASI threads proposal, define technical approach and give opportunity for discussion.

High-level approach

The high-level approach for implementing WASI threads is to create a new module instance for every new thread and only share the memory between created instances.

API

As of today there's a single API that has to be added to the host:

int32_t thread_spawn(void * start_arg)

where start_arg is an opaque pointer and its value should not be inspected by the host environment; instead, it should be passed back to the WASM code. The function return unique non-negative thread identifier integer.
thread_spawn instantiates new thread, and must call the following exported function in a new thread once the thread is ready:

void wasi_thread_start(int thread_id, void *start_arg);

where the start_arg is exactly the same value as the one being passed to the thread_spawn, and thread_id is the same thread ID as the one returned from thread_spawn.

Project structure

We considered two approaches:

• implement proposal in libc-wasi, given it is already part of wasi sdk
• create a new library (in iwasm/core/libraries) that includes the implementation

The main benefit of the former approach is consistency with other APIs; however, we decided to go for the latter approach due to the following reasons:

• WASI threads is still in progress and some of the concepts haven't been fully clarified yet; having it as a separate library help emphasizing the instability of the module to potential users
• conditional compilation of the module won't require too many ifdef's here and there as the whole implementation will live in a separate place; that will enable us to have cleaner and more maintainable code

Compatibility

WAMR already allows for spawning new threads, either using pthread library or wasm_runtime_spawn_thread.

Our goal as part of wasi threads support is to provide a replacement for pthread library (through wasi-libc) therefore we won't guarantee compatibility with pthread library (i.e. both wasi threads and pthread library can not be used at the same time). If we see that using both implementations at the same time might actually be harmful, we might even restrict enabling both features at the same time at build time.
Users who are willing to use threads, should be instructed to use wasi-libc implementation on top of wasi threads in WAMR, so eventually pthread library can be deprecated and removed.

We see a value in keeping wasm_runtime_spawn_thread in parallel to wasi threads as this allows running concurrently multiple instances of a module from the native code (as opposed to spawning new threads from WASM code itself). wasi threads implementation should be able to work in parallel with wasm_runtime_spawn_thread functionality (with caveats, see sections below).

Thread instantiation & management

wasi threads implementation needs some sort of native thread management for multiple reasons, e.g.:

• exception propagation - whenever the exception is being raised (e.g. out of bound memory access or division by zero), all the running threads should be notified and stopped
• there's a discussion whether wasi_thread_exit-like API is needed in Is wasi_thread_exit needed? WebAssembly/wasi-threads#7; if so, or if other syscalls are being added for threads (e.g. for setting thread priorities), thread manager must be able to locate the thread and interact with it accordingly.

WAMR already have a concept of WASM threads implemented using pthread library and thread management using thread manager. We'll use this thread manager for prototype as it gives us out of the box thread instantiation. However, because thread manager also enforces aux stack management, we'll have to refactor the module to allow pass aux stack control to the application.

Auxiliary stack management

Currently WAMR manages stack boundaries when threads are enabled:

• Based on the maximum number of threads, WAMR segments aux stack so segments can be assigned to the thread code
• When a new thread is created (here or here), one of the segments is being assigned to the thread using allocate_aux_stack method.
• When thread finishes, stack segment has to be released using free_aux_stack so new threads can use the segment.

The approach is not ideal for a few reasons:

• stack size, configured e.g. by -z stack-size=XXX is split across all the threads, therefore e.g. the main thread receives only a fraction of the stack size. This is inconsistent with LLD's -Wl,-stack_size, where the thread receives a full stack_size value.
• WAMR's stack allocation might interfere with user space stack allocation; for example, wasi-libc's pthread_create (similarly to musl's pthread_create) allocates memory for the stack dynamically, so there's no need for the host to manage stack for multiple threads.

We'll disable stack allocation for new threads created with wasi_thread_spawn and instead, we off-load stack management to WASM application.

Similar approach can be used for wasm_runtime_spawn_thread; we'll use wasm_runtime_module_malloc/wasm_runtime_module_free to allocate aux stack either in libc heap (if malloc/free are exported) or app heap.

Stack overrun detection

At the moment WAMR has the information about stack boundaries so can detect aux stack overflow/underflow when it happens. This detection still will be possible for wasm_runtime_spawn_thread because the memory for those methods will be allocated in native code. However, WAMR won't have access to stack boundaries defined by wasi-libc's pthread implementation. This was already discussed in WebAssembly/wasi-threads#12; one of the approach is to use binaryen's stack-check pass.

MVP Tasks

• Create a skeleton of the wasi threads library Create a placeholder for WASI threads implementation #1783
• provide implementation for thread_spawn function Add implementation for wasi_thread_spawn() #1786
• refactor thread manager (and possibly interpreters) to enable aux stack management by the application Enable aux stack allocations on application heap #1799
• improve thread ID allocation method to enable re-using unused thread ids and avoid integer overflow Improve WASI thread id allocator to reuse identifiers #1809
• gracefully exit threads when the exception is thrown in one of the threads Exception propagation with WASI threads #1812
• make sure proc_exit terminates all of the threads, and traps kill the process
  • proc_exit and traps do not stop thread executing blocking instructions #1910
  • Possible race condition when trap in spawn thread (using WASI threads) #1909
• build wasi-threads examples in the CI Add CI for WASI threads #1819
• Fast JIT atomic operations implementation
• execute tests (update proposal repository with tests, use existing libc test suite etc)
  • classic interpreter
  • fast interpreter
  • JIT
  • AOT
• add examples
  • add pure wasi examples Create a placeholder for WASI threads implementation #1783
  • add examples using pthread API and wasi-libc
• update documentation

[lum1n0us]

Users who are willing to use threads, should be instructed to use wasi-libc implementation on top of wasi threads in WAMR, so eventually pthread library can be deprecated and removed.

Does it mean users have to write some glue code to satisfy wasi-threads requirements and can't keep their original version of code?

[yamt]

Users who are willing to use threads, should be instructed to use wasi-libc implementation on top of wasi threads in WAMR, so eventually pthread library can be deprecated and removed.

Does it mean users have to write some glue code to satisfy wasi-threads requirements and can't keep their original version of code?

embedder will need wasi initialization etc.

wasm apps will need a rebuild at least.
it might or might not need some code changes to accommodate for incompatibilities between pthread implementations.

[loganek]

Does it mean users have to write some glue code to satisfy wasi-threads requirements and can't keep their original version of code?

Glue code might or might not be needed depending on the features being used. At the moment wasi-libc does not implement all the pthread API (having said that, I don't think WAMR pthread implement it all either?) so there might be some limitations.

I'd suggest keeping pthread as an option for now and only if we achieve feature parity in wasi-libc (or we're close to that, and we're ok accepting potential gaps), we can drop pthread library.

[loganek]

I'm resolving the issue as the functionality has been already implemented and a lot of tests have been added. We'll continue testing and bugfixing the feature but any further problems will be tracked in separate github issue.