Feature: Threading support

[n1474335]

Summary

Operations that take a long time to run currently lock up the entire app. It should be possible to run all the operation logic within a separate thread, allowing the user to continue interacting with the app, even if the recipe is still executing. The user should also be able to cancel a bake that is taking too long or has crashed.

Possible solutions

Web Workers provide rudimentary JavaScript threading within browsers. The entire CyberChef core could sit in a Web Worker and communicate with the web app via message passing. As the core has been designed to operate distinctly from the front-end app, this shouldn't be too complex to implement.

[n1474335]

I've drawn out design diagrams for the current and proposed systems. They show how each of the components gets instantiated in each model, which it is important to understand in order to ensure that there is minimal duplication between the core and app. Note that this does not include Utils.js which is used almost everywhere and will have to be compiled into both the worker and the front-end (in future it may make sense to split Utils.js into two if there isn't much crossover).

Current design diagram

Proposed design diagram

[tlwr]

We've discussed this before: I think it is a good thing to do in general (UI problems,etc), and also very good for the long-term client/server vision (where operations can run in node or another language).

IIRC browser compatibility is pretty good for Web Workers. Do you plan on including a fallback at all?

Looking forward to this!

[n1474335]

Browser support is fairly good and covers all the versions that CyberChef officially supports, so there won't be any fallbacks. If the user is running a browser that doesn't support web workers, the chances are that other parts of the app will also fail anyway.

The roadmap looks roughly like this (no timescales yet):

1. Threading - add Web Workers to support threading of the entire baking process.
2. Modules - only download the code for a core subset of common operations initially, then import the code to run other types of operation as requested. importScripts() should make this possible.
3. Abstract the core - give the user the ability to specify where operations are run, i.e. locally, on a specified server, or maybe even in the "cloud".

[tlwr]

I had never seen importScripts before but now I can imagine how it would make modules much easier to implement. Good find.