-
Notifications
You must be signed in to change notification settings - Fork 2
build-ci v3: Infrastructure Update for spack v1.X
#253
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
21cdda9
Add new spack-checkout-updated-ref action for spack-managed package r…
CodeGat cd6cdbd
Update .gitignore to ignore build secrets
CodeGat 0039903
Dockerfile: Update to spack v1, additional fixes
CodeGat 9fa6bad
ci*.yml: Update to v1, manage spack-packages repos via `spack repo up…
CodeGat cd0aa27
Update compilers for dev and prod to `intel-oneapi-compilers-classic@…
CodeGat 6466d22
compose.[dev|prod].yaml: Added OCI buildcache build secrets, updated …
CodeGat 74282de
Update default access-spack-packages-ref to `api-v2`, spack-ref to `r…
CodeGat 0496cef
Update inter-workflow refs to `v3`
CodeGat 7b7d34a
fix: Update spack-checkout-updated-refs get-git-ref-info action to us…
CodeGat cad4fbe
fix: fetch repository history to get SHA for the new ref, force updat…
CodeGat fde9574
Added specific versions for the compose.prod.yaml
CodeGat 2ef704f
fix: update builtin-spack-packages-ref to develop
CodeGat c1d2bb7
fix: Check that spack is an acceptable version for build-ci@v3, updat…
CodeGat c7fe733
tools: Added mass-pr-create.sh script
CodeGat 59255bb
Update Dockerfile to make secrets optional, move tmate install to end…
CodeGat e4357ac
Update default SPACK_VERSION to `v1.1`
CodeGat a20caaf
Updated compose files to v1.1
CodeGat 5fe9577
Update default spack-ref to releases/v1.1
CodeGat 7a00b7e
Update scope to site rather than user,
CodeGat 62ad825
Continue creating logs even if one spec failed to install
CodeGat 057acb6
Add spack concretize --fresh --force pre install
CodeGat a582fcc
Use updated v1 volume instead v1.0
CodeGat 43ef5da
Update guidance regarding default spack ref
CodeGat 2acb571
Remove --fresh concretization
CodeGat 9225ee9
ci*.yml: Don't explicitly load compilers
CodeGat c0fef23
Add back commented targets for prod/dev image packages
CodeGat e1aa3f3
Update prod image names
CodeGat e4ba85d
Remove redundant if
CodeGat f731479
Remove confusing sleep step in spack-checkout-updated-ref example
CodeGat 2d077fc
Update .github/actions/spack-checkout-updated-ref/README.md
CodeGat cacd818
Update workflow input/output examples
CodeGat File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| # Update Existing Repo and Checkout Ref | ||
|
|
||
| Action that updates an existing repository, and checks out the updated ref. | ||
|
|
||
| ## Inputs | ||
|
|
||
| | Name | Type | Description | Required | Default | Example | | ||
| | ---- | ---- | ----------- | -------- | ------- | ------- | | ||
| | `spack-packages-repository-name` | `string` | The name of the repository used by spack to update and checkout the ref (given in the spack config file `repos.yaml`) | `true` | N/A | `"builtin"` | | ||
| | `spack-packages-repository-path` | `string` (path) | The path to the repository to update and check out the ref | `true` | N/A | `"/root/.spack/package_repos/fncqgg4/repos/spack_repo/builtin"` | | ||
| | `ref` | `string` (git branch, tag or sha) | The git ref to check out | `true` | N/A | `"main"` or `"v1"` or `"f8r73g3"` | | ||
| | `spack-instance-root-path` | `string` (path) | The path to the spack instance root, used to setup the spack environment | `true` | N/A | `"/opt/spack"` | | ||
| An example [`repos.yaml` file](https://github.com/ACCESS-NRI/spack-config/blob/main/common-api-v2/repos.yaml) as referenced above. | ||
| ## Outputs | ||
|
|
||
| | Name | Type | Description | Example | | ||
| | ---- | ---- | ----------- | ------- | | ||
| | `sha` | `string` (sha) | The SHA of the checked out ref | `"5a1cdc4e4617fcd6ba1cccf1cd0432b5631983be"` | | ||
| | `updated` | `string` (boolean) | Whether there was actually an update to the ref | `"true"` or `"false"` | | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Simple | ||
|
|
||
| ```yaml | ||
| # ... | ||
| jobs: | ||
| update-repo: | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| SPACK_ROOT: /opt/spack | ||
| steps: | ||
| - id: repo | ||
| run: | | ||
| . ${{ env.SPACK_ROOT }}/share/spack/setup-env.sh | ||
| echo "path=$(spack location --repo builtin)" >> $GITHUB_OUTPUT | ||
|
|
||
| - id: update | ||
| uses: ./.github/actions/spack-checkout-updated-ref | ||
| with: | ||
| spack-packages-repository-name: builtin | ||
| spack-packages-repository-path: ${{ steps.repo.outputs.path }} | ||
| ref: develop | ||
| spack-instance-root-path: ${{ env.SPACK_ROOT }} | ||
|
|
||
| - run: echo "The builtin spack-packages repo was updated to ${{ steps.update.outputs.sha }}" | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| name: Checkout Ref Via Spack | ||
| description: Updates and checks out a given ref for an existing spack-packages repository via spack | ||
| inputs: | ||
| spack-packages-repository-name: | ||
| description: | | ||
| The spack name of the repository to update and check out the ref. | ||
| This is the repos.NAME key in spacks repos.yaml config file - eg. builtin | ||
| required: true | ||
| spack-packages-repository-path: | ||
| description: The path to the repository to update and check out the ref | ||
| required: true | ||
| ref: | ||
| description: The git ref to check out | ||
| required: true | ||
| spack-instance-root-path: | ||
| description: The path to the spack instance root, used to setup the spack environment | ||
| required: true | ||
| outputs: | ||
| sha: | ||
| description: 'The SHA of the checked out ref' | ||
| value: ${{ steps.ref-sha.outputs.sha }} | ||
| updated: | ||
| description: 'Whether the repository was updated' | ||
| value: ${{ steps.post-update.outputs.updated }} | ||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Get initial SHA | ||
| id: initial | ||
| shell: bash | ||
| run: | | ||
| sha=$(git -C ${{ inputs.spack-packages-repository-path }} rev-parse HEAD^{}) | ||
| echo "${{ inputs.spack-packages-repository-name }} Initial SHA: $sha" | ||
|
|
||
| echo "sha=$sha" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Fetch latest for repo | ||
| shell: bash | ||
| run: git -C ${{ inputs.spack-packages-repository-path }} fetch --force --tags | ||
|
|
||
| - name: Get SHA for ref | ||
| id: ref-sha | ||
| uses: access-nri/actions/.github/actions/get-git-ref-info@main | ||
| with: | ||
| repository-path: ${{ inputs.spack-packages-repository-path }} | ||
| ref: ${{ inputs.ref }} | ||
|
|
||
| - name: Update ref via Spack | ||
| shell: bash | ||
| id: update | ||
| continue-on-error: true | ||
| run: | | ||
| . ${{ inputs.spack-instance-root-path }}/share/spack/setup-env.sh | ||
| spack repo update ${{ inputs.spack-packages-repository-name }} --commit ${{ steps.ref-sha.outputs.sha }} | ||
|
|
||
| - name: Force Update ref via Git if Failure | ||
| if: steps.update.outcome == 'failure' | ||
| shell: bash | ||
| # FIXME: If there is ever a spack repo update --force option, use that for the above command and delete this step | ||
| run: git -C ${{ inputs.spack-packages-repository-path }} checkout --force ${{ steps.ref-sha.outputs.sha }} | ||
|
|
||
| - name: Check if updated | ||
| id: post-update | ||
| shell: bash | ||
| run: | | ||
| if [ "${{ steps.initial.outputs.sha }}" != "${{ steps.ref-sha.outputs.sha }}" ]; then | ||
| echo "Repository was updated from ${{ steps.initial.outputs.sha }} to ${{ inputs.ref }} (${{ steps.ref-sha.outputs.sha }})" | ||
| echo "updated=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "Repository was not updated, stayed at ${{ inputs.ref }} (${{ steps.initial.outputs.sha }})" | ||
| echo "updated=false" >> $GITHUB_OUTPUT | ||
| fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.