Regarding:
Probably a bigger question than this workflow, but: Once CARB has all the files we've delivered them onto their own servers, it should conceptually be possible for the whole workflow to run without any S3 access. How many changes would be needed at the CLI level to achieve that?
Originally posted by @infotroph in #18 (comment)
We will need to ensure that any data logistics which derives from s3 data artifacts occurs as its own step, such that is can be not-run if all artifacts are local.
Run preparation steps should occur in a separate verb.
Regarding:
Originally posted by @infotroph in #18 (comment)
We will need to ensure that any data logistics which derives from s3 data artifacts occurs as its own step, such that is can be not-run if all artifacts are local.
Run preparation steps should occur in a separate verb.