Current behaviour
For a CERN deployment, when a researcher uses an image such as:
docker.io/myorg/myimage:mytag
it can happen that the workflow execution failes because REANA is not able to pull this image due to rate limits:
Container job failed, error: failed to pull and unpack image "docker.io/myorg/myimage:mytag": failed to copy:
httpReadSeeker: failed open: unexpected status code https://... 429 Too Many Requests -
Server message: toomanyrequests:
You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
In order to avoid Docker Hub pull rate limit, the researchers can profit from the CERN Registry's pull-through cache and specify their container image like:
registry.cern.ch/docker.io/myorg/myimage:mytag
This works well, however it requires manual editing of the workflow specification, which may not be always practical.
Expected behaviour
It may be good to offer a cluster-wide customisation option via a Helm value (for example, an environment variable`USE_CERN_REGISTRY_PULL_THROGUH_CACHE that would be false by default) that would switch this behaviour globally for all user workloads.
That is, when REANA sees a Docker Hub image location in the workflow specification and when this variable is on, REANA would silently add the leading registry.cern.ch/ to the FQIN specified by the user in order to profit from the pull-through cache, so that users do not have to do anything in their workflow definitions at all.
Notes
This is most important for Docker Hub, but see also quay, ghcr, etc that the CERN Registry supports.
Current behaviour
For a CERN deployment, when a researcher uses an image such as:
it can happen that the workflow execution failes because REANA is not able to pull this image due to rate limits:
In order to avoid Docker Hub pull rate limit, the researchers can profit from the CERN Registry's pull-through cache and specify their container image like:
This works well, however it requires manual editing of the workflow specification, which may not be always practical.
Expected behaviour
It may be good to offer a cluster-wide customisation option via a Helm value (for example, an environment variable`USE_CERN_REGISTRY_PULL_THROGUH_CACHE that would be false by default) that would switch this behaviour globally for all user workloads.
That is, when REANA sees a Docker Hub image location in the workflow specification and when this variable is on, REANA would silently add the leading
registry.cern.ch/to the FQIN specified by the user in order to profit from the pull-through cache, so that users do not have to do anything in their workflow definitions at all.Notes
This is most important for Docker Hub, but see also quay, ghcr, etc that the CERN Registry supports.