Workflow params [e2e prod]

Signed-off-by: Ben Sherman <bentshermann@gmail.com>

Author: bentsherman

docs/migrations/25-10.md

@@ -8,6 +8,31 @@ This page summarizes the upcoming changes in Nextflow 25.10, which will be relea
 This page is a work in progress and will be updated as features are finalized. It should not be considered complete until the 25.10 release.
 :::
 
+## New features
+
+<h3>Workflow params</h3>
+
+The `params` block is a new way to declare pipeline parameters in a Nextflow script:
+
+```nextflow
+params {
+  // Path to input data.
+  input: Path
+
+  // Whether to save intermediate files.
+  save_intermeds: Boolean = false
+}
+
+workflow {
+  println "params.input = ${params.input}"
+  println "params.save_intermeds = ${params.save_intermeds}"
+}
+```
+
+This syntax allows you to declare all parameters in one place with explicit type annotations, and it allows Nextflow to validate parameters at runtime.
+
+See {ref}`workflow-params-def` for details.
+
 ## Enhancements
 
 <h3>New syntax for workflow handlers</h3>
@@ -74,4 +99,6 @@ This feature addresses previous inconsistencies in timestamp representations.
 
 ## Deprecations
 
+- The legacy type detection of CLI parameters is disabled when using the strict syntax (`NXF_SYNTAX_PARSER=v2`). {ref}`Legacy parameters <workflow-params-legacy>` in the strict syntax should not rely on legacy type detection. Alternatively, use the new `params` block to convert CLI parameters based on their type annotations. Legacy type detection can be disabled globally by setting the environment variable `NXF_DISABLE_PARAMS_TYPE_DETECTION=true`.
+
 - The use of workflow handlers in the configuration file has been deprecated. You should define workflow handlers in the pipeline script or a plugin instead. See {ref}`config-workflow-handlers` for details.

docs/reference/syntax.md

@@ -28,7 +28,8 @@ A Nextflow script may contain the following top-level declarations:
 - Shebang
 - Feature flags
 - Include declarations
-- Parameter declarations
+- Params block
+- Parameter declarations (legacy)
 - Workflow definitions
 - Process definitions
 - Function definitions
@@ -107,9 +108,22 @@ The following definitions can be included:
 - Processes
 - Named workflows
 
-### Parameter
+### Params block
 
-A parameter declaration is an assignment. The target should be a pipeline parameter and the source should be an expression:
+The params block consists of one or more *parameter declarations*. A parameter declaration consists of a name and an optional default value:
+
+```nextflow
+params {
+    input: Path
+    save_intermeds: Boolean = false
+}
+```
+
+Only one params block may be defined in a script.
+
+### Parameter (legacy)
+
+A legacy parameter declaration is an assignment. The target should be a pipeline parameter and the source should be an expression:
 
 ```nextflow
 params.message = 'Hello world!'

docs/vscode.md

@@ -26,7 +26,7 @@ The language server parses scripts and config files according to the {ref}`Nextf
 
 When you hover over certain source code elements, such as variable names and function calls, the extension provides a tooltip with related information, such as the definition and/or documentation for the element.
 
-If a [Javadoc](https://en.wikipedia.org/wiki/Javadoc) comment is defined above a workflow, process, or function, the extension will include the contents of the comment in hover hints. The following is an example Javadoc comment:
+If a [Javadoc](https://en.wikipedia.org/wiki/Javadoc) comment is defined above a workflow, process, function, or parameter in a `params` block, the extension will include the contents of the comment in hover hints. The following is an example Javadoc comment:
 
 ```nextflow
 /**

docs/workflow.md

@@ -22,7 +22,60 @@ workflow {
 }
 ```
 
-### Parameters
+(workflow-params-def)=
+
+## Parameters
+
+Parameters can be declared in a Nextflow script with the `params` block or with *legacy* parameter declarations.
+
+### Params block
+
+:::{versionadded} 25.10.0
+:::
+
+:::{note}
+This feature requires the {ref}`strict syntax <strict-syntax-page>` to be enabled (`NXF_SYNTAX_PARSER=v2`).
+:::
+
+A script can declare parameters using the `params` block:
+
+```nextflow
+params {
+    // Path to input data.
+    input: Path
+
+    // Whether to save intermediate files.
+    save_intermeds: Boolean = false
+}
+```
+
+The following types can be used for parameters:
+
+- {ref}`stdlib-types-boolean`
+- {ref}`stdlib-types-float`
+- {ref}`stdlib-types-integer`
+- {ref}`stdlib-types-path`
+- {ref}`stdlib-types-string`
+
+Parameters can be used in the entry workflow:
+
+```nextflow
+workflow {
+    analyze(params.input, params.save_intermeds)
+}
+```
+
+:::{note}
+As a best practice, parameters should only be used directly in the entry workflow and passed to workflows and processes as explicit inputs.
+:::
+
+The default value can be overridden by the command line, params file, or config file. Parameters from multiple sources are resolved in the order described in {ref}`cli-params`. Parameters specified on the command line are converted to the appropriate type based on the corresponding type annotation.
+
+A parameter that doesn't specify a default value is a *required* param. If a required param is not given a value at runtime, the run will fail.
+
+(workflow-params-legacy)=
+
+### Legacy parameters
 
 Parameters can be declared by assigning a `params` property to a default value:
 
@@ -38,10 +91,6 @@ workflow {
 }
 ```
 
-:::{note}
-As a best practice, `params` should be used only in the entry workflow and passed to workflows and processes as explicit inputs.
-:::
-
 The default value can be overridden by the command line, params file, or config file. Parameters from multiple sources are resolved in the order described in {ref}`cli-params`.
 
 ## Named workflows

modules/nextflow/src/main/groovy/nextflow/Session.groovy

@@ -120,6 +120,16 @@ class Session implements ISession {
      */
     ScriptBinding binding
 
+    /**
+     * Params that were specified on the command line.
+     */
+    Map cliParams
+
+    /**
+     * Params that were specified in the configuration.
+     */
+    Map configParams
+
     /**
      * Holds the configuration object
      */
@@ -425,7 +435,7 @@ class Session implements ISession {
     /**
      * Initialize the session workDir, libDir, baseDir and scriptName variables
      */
-    Session init( ScriptFile scriptFile, List<String> args=null ) {
+    Session init( ScriptFile scriptFile, List<String> args=null, Map<String,?> cliParams=null, Map<String,?> configParams=null ) {
 
         if(!workDir.mkdirs()) throw new AbortOperationException("Cannot create work-dir: $workDir -- Make sure you have write permissions or specify a different directory by using the `-w` command line option")
         log.debug "Work-dir: ${workDir.toUriString()} [${FileHelper.getPathFsType(workDir)}]"
@@ -452,6 +462,8 @@ class Session implements ISession {
         this.workflowMetadata = new WorkflowMetadata(this, scriptFile)
 
         // configure script params
+        this.cliParams = cliParams
+        this.configParams = configParams
         binding.setParams( (Map)config.params )
         binding.setArgs( new ScriptRunner.ArgsList(args) )
 

modules/nextflow/src/main/groovy/nextflow/cli/CmdRun.groovy

@@ -325,12 +325,18 @@ class CmdRun extends CmdBase implements HubOptions {
         // -- specify the arguments
         final scriptFile = getScriptFile(pipeline)
 
+        // -- load command line params
+        final baseDir = scriptFile.parent
+        final cliParams = parsedParams(ConfigBuilder.getConfigVars(baseDir))
+
         // create the config object
         final builder = new ConfigBuilder()
                 .setOptions(launcher.options)
                 .setCmdRun(this)
-                .setBaseDir(scriptFile.parent)
-        final config = builder .build()
+                .setBaseDir(baseDir)
+                .setCliParams(cliParams)
+        final config = builder.build()
+        final configParams = builder.getConfigParams()
 
         // check DSL syntax in the config
         launchInfo(config, scriptFile)
@@ -376,7 +382,7 @@ class CmdRun extends CmdBase implements HubOptions {
         }
 
         // -- run it!
-        runner.execute(scriptArgs, this.entryName)
+        runner.execute(scriptArgs, cliParams, configParams, this.entryName)
     }
 
     protected void printBanner() {
@@ -698,7 +704,7 @@ class CmdRun extends CmdBase implements HubOptions {
     }
 
     static protected parseParamValue(String str) {
-        if ( SysEnv.get('NXF_DISABLE_PARAMS_TYPE_DETECTION') )
+        if ( SysEnv.get('NXF_DISABLE_PARAMS_TYPE_DETECTION') || NF.isSyntaxParserV2() )
             return str
 
         if ( str == null ) return null

modules/nextflow/src/main/groovy/nextflow/config/ConfigBuilder.groovy

@@ -58,6 +58,8 @@ class ConfigBuilder {
 
     Path currentDir
 
+    Map<String,?> cliParams
+
     boolean showAllProfiles
 
     String profile = DEFAULT_PROFILE
@@ -78,9 +80,11 @@ class ConfigBuilder {
 
     Map<String,String> env = new HashMap<>(SysEnv.get())
 
-    List<String> warnings = new ArrayList<>(10);
+    List<String> warnings = new ArrayList<>(10)
+
+    Map<String,Object> declaredParams = [:]
 
-    {
+    ConfigBuilder() {
         setHomeDir(Const.APP_HOME_DIR)
         setCurrentDir(Paths.get('.'))
     }
@@ -111,6 +115,11 @@ class ConfigBuilder {
         return this
     }
 
+    ConfigBuilder setCliParams( Map<String,?> cliParams ) {
+        this.cliParams = cliParams
+        return this
+    }
+
     ConfigBuilder setBaseDir( Path path ) {
         this.baseDir = path.complete()
         return this
@@ -159,6 +168,10 @@ class ConfigBuilder {
         return this
     }
 
+    Map<String,Object> getConfigParams() {
+        return declaredParams
+    }
+
     static private wrapValue( value ) {
         if( !value )
             return ''
@@ -324,11 +337,11 @@ class ConfigBuilder {
         // this is needed to make sure to reuse the same
         // instance of the config vars across different instances of the ConfigBuilder
         // and prevent multiple parsing of the same params file (which can even be remote resource)
-        return cacheableConfigVars(baseDir)
+        return getConfigVars(baseDir)
     }
 
     @Memoized
-    static private Map cacheableConfigVars(Path base) {
+    static Map getConfigVars(Path base) {
         final binding = new HashMap(10)
         binding.put('baseDir', base)
         binding.put('projectDir', base)
@@ -348,8 +361,8 @@ class ConfigBuilder {
                 .setIgnoreIncludes(ignoreIncludes)
         ConfigObject result = new ConfigObject()
 
-        if( cmdRun && (cmdRun.hasParams()) )
-            parser.setParams(cmdRun.parsedParams(configVars()))
+        if( cliParams )
+            parser.setParams(cliParams)
 
         // add the user specified environment to the session env
         env.sort().each { name, value -> result.env.put(name,value) }
@@ -380,7 +393,7 @@ class ConfigBuilder {
             }
 
             if( validateProfile ) {
-                checkValidProfile(parser.getProfiles())
+                checkValidProfile(parser.getDeclaredProfiles())
             }
 
         }
@@ -414,6 +427,7 @@ class ConfigBuilder {
         final config = parse0(parser, entry)
         if( NF.getSyntaxParserVersion() == 'v1' )
             validate(config, entry)
+        declaredParams.putAll(parser.getDeclaredParams())
         result.merge(config)
     }
 
@@ -723,8 +737,8 @@ class ConfigBuilder {
         }
 
         // -- add the command line parameters to the 'taskConfig' object
-        if( cmdRun.hasParams() )
-            config.params = mergeMaps( (Map)config.params, cmdRun.parsedParams(configVars()), NF.strictMode )
+        if( cliParams )
+            config.params = mergeMaps( (Map)config.params, cliParams, NF.strictMode )
 
         if( cmdRun.withoutDocker && config.docker instanceof Map ) {
             // disable docker execution

modules/nextflow/src/main/groovy/nextflow/config/ConfigParser.groovy

@@ -67,6 +67,11 @@ interface ConfigParser {
      */
     ConfigParser setParams(Map vars)
 
+    /**
+     * Set the profiles that should be applied.
+     */
+    ConfigParser setProfiles(List<String> profiles)
+
     /**
      * Parse a config object from the given source.
      */
@@ -75,13 +80,13 @@ interface ConfigParser {
     ConfigObject parse(Path path)
 
     /**
-     * Set the profiles that should be applied.
+     * Get the set of declared profiles.
      */
-    ConfigParser setProfiles(List<String> profiles)
+    Set<String> getDeclaredProfiles()
 
     /**
-     * Get the set of available profiles.
+     * Get the map of declared params.
      */
-    Set<String> getProfiles()
+    Map<String,Object> getDeclaredParams()
 
 }

modules/nextflow/src/main/groovy/nextflow/config/parser/v1/ConfigParserV1.groovy

@@ -123,10 +123,15 @@ class ConfigParserV1 implements ConfigParser {
     }
 
     @Override
-    Set<String> getProfiles() {
+    Set<String> getDeclaredProfiles() {
         Collections.unmodifiableSet(conditionalNames)
     }
 
+    @Override
+    Map<String,Object> getDeclaredParams() {
+        [:]
+    }
+
     private Grengine getGrengine() {
         if( grengine ) {
             return grengine