Goal: Capture all relevant output from each run into a [set of] log file[s] that can be preserved as an artifact.
Currently approximated by telling people to redirect their console output to a log file, with output lost if they forget.
Proposed design (paraphrased from a Slack thread with @hdpriest-ui; please correct if I mangled anything) : Top-level CLI command (magic-ensemble) copies all console output that it sees into a standard location ([rundir]/logs?). The "that the CLI sees" part matters because substeps, especially ones that are run in parallel or on distributed resources, may also define their own logs at locations that are configurable but default to being inside the rundir.
Order of implementation: Start with top-level log to catch whatever bubbles up to it, then refine lower-level steps.
Goal: Capture all relevant output from each run into a [set of] log file[s] that can be preserved as an artifact.
Currently approximated by telling people to redirect their console output to a log file, with output lost if they forget.
Proposed design (paraphrased from a Slack thread with @hdpriest-ui; please correct if I mangled anything) : Top-level CLI command (
magic-ensemble) copies all console output that it sees into a standard location ([rundir]/logs?). The "that the CLI sees" part matters because substeps, especially ones that are run in parallel or on distributed resources, may also define their own logs at locations that are configurable but default to being inside the rundir.Order of implementation: Start with top-level log to catch whatever bubbles up to it, then refine lower-level steps.