NodeJS API

[d98762625]

Move NodeJS API into it's own issue. First mentioned in PR #284

---

We should aim to be able to export specific modules from the CyberChef package so that the 200+ operations can be used effectively in other projects.

An example (exact API still open to suggestions):

import { base64, base32 } from "cyberchef";

base64.from("SGVsbG8sIFdvcmxkIQ==");
base32.to("some input");

Also working with commonJS require.

[d98762625]

Some options on the API:

If we have the functions in the format chef.base32.from:

We can expose the operation's properties there, or we can just make that the alias for the run function.

So if we expose the properties, we will have

chef.base32.from.inputType // => 'string'
chef.base32.from.outputType // => 'byteArray'
chef.base32.from.args // => [...]
etc

Which could make the API more self-documenting, for people using it in a REPL interface. BUT it does mean that to run the operation you need

chef.base32.from.run(...)

Or we just expose the run function, giving

chef.base32.from(...)

With both options, we can still be as lenient as possible with input types by wrapping the operation's run function in some type conversion.

---

I think just exposing run would be cleaner, I'm just thinking about how a user would explore input/output types & other operation properties on the library to be able to build more complex chains of operations.

If we just expose the run function, we could expose a findType function on the library which would duck type some given value for you.

Also, the dish.get function translates input types for these operations, allowing us to be tolerant on input types in the library. But it does mean that every operation in the node library will be async.

@n1474335 thoughts?

[n1474335]

Translating types

I hadn't considered that this would make everything async. In ES7, I don't think it matters a huge amount as async/await make it very easy to deal with this, however I remember asynchronous programming being a bit of a pain when I first started, so I'd be keen to minimise it as much as possible for the benefit of those just starting out.

I don't really have any solutions to that at the moment. The Dish._translate() function has to be async as it sometimes has to load files. I guess for now we just have to make it all async and if we can come up with a better solution, we can address it later.

API

Now that I see the API laid out, I'm wondering if we should just go back to the idea of exposing each op separately, rather than trying to bunch them together into related namespaces. This looks quite simple and intuitive to me:

chef.fromBase64(...)

chef.toHex(...)

chef.entropy(...)

chef.convertArea(...)

chef.gzip(...)

We could export the chef object, but also all the separate operations on their own. That way people can still get the benefits of ES6 tree-shaking if they want it.

Exposing the configuration details would be useful, especially the descriptions and arguments. Perhaps we could have a separate lookup function that returns the metadata. We could even have a search function that works in a similar way to the search bar in the web app:

chef.info("To Base64")
// => { name: "To Base64", description: "...", inputType: "byteArray", ... }

chef.search("xor")
// => [{ name: "XOR", desc...}, { name: "XOR Brute Force", desc...}, ...]

One of the returned properties could even be a reference to the run() function, so you could do something like:

const userStr = getInput("Please enter your search: ");

chef.search(userStr)[0].run(...);

[mattnotmitt]

Are we intending to use Promises for this?

[n1474335]

I think we'll have to if it's async. I'd rather not as they can be confusing to people with limited JavaScript experience, but if one of the possible data types is File, I think we have to operate asynchronously as the FileReader API is async.

[n1474335]

In ES7, it's as simple as this though:

await chef.toBase64(...)

Granted, you then have to declare your scope as async.

[d98762625]

@n1474335 I agree with you on the per-operation API. I don't think there's much benefit in grouping operations (base32.from, base32.to) - especially for the amount of extra it adds in manually declaring the relationships.

I like the info and search - I will play around with those.

@artemisbot async functions return promises anyway, so can use the promise syntax with these if you don't want to use async/await.

[d98762625]

We can also support a callback API, adding flexibility. See example (taken from a Google example):

/**
 * Create and send request to Google API
 * @param parameters Parameters used to form request
 * @param callback   Callback when request finished or error found
 */

export function createAPIRequest(parameters, callback) {
  if (callback) {
    createAPIRequestAsync(parameters)
        .then(r => callback(null, r))
        .catch(e => callback(e));
  } else {
    return createAPIRequestAsync(parameters);
  }
}

[d98762625]

Some additional details on library API behaviour:

Error handling

Expected errors from operations are currently returned as a string to be outputted in the web UI. Only Unexpected errors are caught by Recipe and are shown in an alert. For the library API, we want these expected errors also to be thrown, under it's own error type (we could make an InputError type for example). These would then conform with the try/catch / .then()/.catch() / (err, result) => {} interface that people would expect and we can encourage handling of this new error type.

In the web logic, we can throw errors from operations then catch our new InputError separately in Recipe and just print out the error, like we do now. @n1474335 Are there any other places that we handle Operation errors?

Output objects - translating types

We will expose a translate function on the chef module to translate outputs of functions, like so:

result = chef.toBase32('input')
file = chef.translate(result, chef.types.LIST_FILE) 

This is like the type conversion @n1474335 has now added to Chef.js as getDishAs (#284), so I will see if I can reuse this.

[n1474335]

@d98762625 This can probably be closed now, can't it? Just trying to clean some things up.

[d98762625]

Yep, I'll close it.