Allow Decision Points / Control Statements in Workflows

[blankenberg]

Some things that would be nice to have available:

• Assertions (Assertion checking in workflows #3954)
• Branching (and rejoining) based upon
  • the result of a previous input step: final job state, dataset content, etc (could be done as part of assertions)
  • user selection at time of workflow execution (allow one workflow to handle several [minorly] different cases)
• Looping:
  • set number of times
  • break based upon assertions

Some other things to consider, although perhaps at the tool level vs. only workflow level

• Allow dataset content to drive/set parameter values, e.g. text value in a file used as value for an input parameter

[jmchilton]

Despite the push back I received the last time we discussed this - my opinion remains that we should to have a expression language baked into Galaxy to reason about both parameters and metadata related inputs before continuing with either branching and looping. It is less obvious that assertions require this - but I believe it would make sense to use the same language (both in terms of an executable backend programming language and visual language for the GUI) in place for assertions.

The common workflow language is discussing an implementation for this as well (xref common-workflow-language/common-workflow-language#494) with comments both from people with concrete use cases and platforms that have previous implemented this (Taverna in particular).

Allow dataset content to drive/set parameter values, e.g. text value in a file used as value for an input parameter

This I believe is a different issue to some degree - we are very close to having this on the backend in some ways. Certainly the CWL branch of Galaxy supports this (https://github.com/common-workflow-language/galaxy). See the commit common-workflow-lab@f2c955b in particular. Simple values connected this way wouldn't require an expression language per se but one can imagine a lot more expressivity using that language. The counter argument perhaps is that encoding expression logic as tabular transformations using Galaxy tools has proven a valuable technique for collection operations and could be applied in this context and may seem more "Galaxy"ish. Regardless though, for this to be usable via workflows constructed in the editor this requires being able to connect to non-inputs - a big wish of mine for sometime.