Config parser (and loader)

[bentsherman]

Close #2723

Implements the strict config syntax in Nextflow, using the shared "compiler" module from the language server.

To use the strict config syntax, simply set NXF_SYNTAX_PARSER=v2 in your environment when running nextflow.

This config loader is much simpler and easier to read. There is no more dependence on metaclasses, instead there is a simple builder DSL for executing a config file and producing a map.

NOTE: While this PR uses Jitpack to load the shared module, this dependency should be inverted before the release of 25.04. That is, eventually the compiler module should reside in Nextflow and the language server should consume it.

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	e6b4048
🔍 Latest deploy log	https://app.netlify.com/sites/nextflow-docs-staging/deploys/67bdf93b441a7c0008156a33

[bentsherman]

Research updates:

• Groovy compiler provides an interface to use a custom parser (in order to ease the transition from Antlr2 to Antlr4), this can be used to inject our custom parser without much hassle
• Config schema effort can be pursued independently of the parser, see Config schema #4201
• Might be able to use the Java security manager to prevent execution of unsafe code in the config file such as file I/O, subprocesses

[bentsherman]

Here's an example to illustrate how the config parser "executes" the config file. Given this config:

foo.bar = 'hello'

foo {
  bar = 'hello'
}

includeConfig 'foobar.config'

The parser transforms this code into the following:

assign(['foo', 'bar'], 'hello')

block('foo', {
  assign(['bar'], 'hello')
}

includeConfig('foobar.config')

Basically each statement is translated to a DSL method (see ConfigDsl). This "hidden DSL" allows us to control how each statement is executed. In other words, the config parser + hidden DSL implement a simple interpreter for the Nextflow config language, separating the config language from the config execution.

I think this is a good model for how we might separate the Nextflow language from the runtime. Instead of hooking directly into Groovy methods and relying on Groovy MOP, metaclass, etc, we can introduce a hidden DSL which allows us to better control how a given statement (e.g. invoking a process in a workflow) is executed by the runtime.

[bentsherman]

Java security manager is being removed. I will keep an eye out for other sandboxing techniques, but in the meantime, a simple solution is to provide a "safe" execution mode where assignment expressions are wrapped in string literals.

[ewels]

Really like the way that this is going, great work 👏🏻

[bentsherman]

@pditommaso this test is related to #1792 . We determined that config profiles are applied this way with the legacy parser. We could patch it, but I figure there's no point now, so just added a test to verify it's behavior.

[bentsherman]

@pditommaso tests are passing, let's review and merge

[christopher-hakkaart]

One minor suggestion but otherwise the docs look great.

[pditommaso]

Awesome