Skip to content

Commit da224f1

Browse files
jakelishmanmergify[bot]
jakelishman
authored and
mergify[bot]
committed
Add GitHub Actions documentation-deployment pipeline (#10610)
* Add GitHub Actions documentation-deployment pipeline This brings a documentation-deployment pipeline into the Qiskit/Terra repository, allowing it to fully deploy its documentation to `qiskit.org`, a task previously only the metapackage could perform. This does not fully unify the documentation with files from the metapackage, it just adds a pipeline to do the final deployment. This includes a revitalised translatable-strings pipeline, which was previously broken on the metapackage for the last month or two. It also previously included a fair amount of legacy weight that was no longer relevant. * Add missing secret insertions * Improve logic for deployments This changes the logic for the deployments so that pushes to 'stable/*' no longer trigger any deployment to qiskit.org. Instead, tag events trigger a deployment to the relevant stable branch, and a tag event of the _latest_ tag triggers a deployment to the documentation root. The translatables logic is modified to push only the latest full-release tag. (cherry picked from commit 80e95d1) # Conflicts: # docs/conf.py
1 parent 7128b27 commit da224f1

11 files changed

Lines changed: 340 additions & 30 deletions

‎.azure/docs-linux.yml‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -19,16 +19,14 @@ jobs:
1919
versionSpec: '${{ parameters.pythonVersion }}'
2020
displayName: 'Use Python ${{ parameters.pythonVersion }}'
2121

22-
- bash: |
23-
set -e
24-
python -m pip install --upgrade pip setuptools wheel
25-
python -m pip install -U "tox<4.4.0"
26-
sudo apt-get update
27-
sudo apt-get install -y graphviz pandoc
22+
- bash: tools/install_ubuntu_docs_dependencies.sh
2823
displayName: 'Install dependencies'
2924

3025
- bash: |
31-
tox -edocs
26+
set -e
27+
tox -e docs
28+
# Clean up Sphinx detritus.
29+
rm -rf docs/_build/html/{.doctrees,.buildinfo}
3230
displayName: 'Run Docs build'
3331
3432
- task: ArchiveFiles@2

‎.azure/tutorials-linux.yml‎

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -16,14 +16,10 @@ jobs:
1616
versionSpec: '${{ parameters.pythonVersion }}'
1717
displayName: 'Use Python ${{ parameters.pythonVersion }}'
1818

19-
- bash: |
20-
set -e
21-
python -m pip install --upgrade pip setuptools wheel
22-
python -m pip install -U "tox<4.4.0"
23-
sudo apt-get update
24-
sudo apt-get install -y graphviz pandoc
19+
- bash: tools/install_ubuntu_docs_dependencies.sh
2520
displayName: 'Install dependencies'
2621

22+
# Sync with '.github/workflows/docs_deploy.yml'
2723
- bash: tools/prepare_tutorials.bash algorithms circuits circuits_advanced operators
2824
displayName: 'Download current tutorials'
2925

‎.github/workflows/docs_deploy.yml‎

Lines changed: 266 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,266 @@
1+
name: Documentation
2+
on:
3+
push:
4+
branches:
5+
- main
6+
tags:
7+
# Only match non-prerelease tags.
8+
- '[0-9]+.[0-9]+.[0-9]'
9+
workflow_dispatch:
10+
inputs:
11+
deploy_prefix:
12+
description: "Deployment prefix (leave blank for the root): https://qiskit.org/documentation/<prefix>."
13+
required: false
14+
type: string
15+
do_deployment:
16+
description: "Push to qiskit.org?"
17+
required: false
18+
type: boolean
19+
do_translatables:
20+
description: "Push translatable strings?"
21+
required: false
22+
type: boolean
23+
24+
jobs:
25+
build:
26+
if: github.repository_owner == "Qiskit"
27+
name: Build
28+
runs-on: ubuntu-latest
29+
30+
outputs:
31+
latest_tag: ${{ steps.latest_tag.outputs.latest_tag }}
32+
33+
steps:
34+
- uses: actions/checkout@v3
35+
with:
36+
# We need to fetch the whole history so 'reno' can do its job and we can inspect tags.
37+
fetch-depth: 0
38+
39+
- name: Determine latest full release tag
40+
id: latest_tag
41+
run: |
42+
set -e
43+
latest_tag=$(git tag --list --sort=-version:refname | sed -n '/^[0-9]\+\.[0-9]\+\.[0-9]\+$/p' | head -n 1)
44+
echo "Latest release tag: '$latest_tag'"
45+
echo "latest_tag=$latest_tag" >> "$GITHUB_OUTPUT"
46+
47+
- uses: actions/setup-python@v4
48+
name: Install Python
49+
with:
50+
# Sync with 'documentationPythonVersion' in 'azure-pipelines.yml'.
51+
python-version: '3.9'
52+
53+
- name: Install dependencies
54+
run: tools/install_ubuntu_docs_dependencies.sh
55+
56+
# Sync with '.azure/tutorials-linux.yml'.
57+
- name: Download current tutorials
58+
run: tools/prepare_tutorials.bash algorithms circuits circuits_advanced operators
59+
shell: bash
60+
61+
# This is just to have tox create the environment, so we can use it to execute the tutorials.
62+
# We want to re-use it later for the build, hence 'tox run --notest' instead of 'tox devenv'.
63+
- name: Prepare Python environment
64+
run: tox run -e docs --notest
65+
66+
# The reason to use the custom script rather than letting 'nbsphinx' do its thing normally
67+
# within the Sphinx build is so that the execution process is the same as in the test CI.
68+
- name: Execute tutorials in place
69+
run: .tox/docs/bin/python tools/execute_tutorials.py docs/tutorials
70+
env:
71+
QISKIT_CELL_TIMEOUT: "300"
72+
73+
- name: Build documentation
74+
# We can skip re-installing the package, since we just did it a couple of steps ago.
75+
run: tox run -e docs --skip-pkg-install
76+
env:
77+
QISKIT_ENABLE_ANALYTICS: "true"
78+
# We've already built them.
79+
QISKIT_DOCS_BUILD_TUTORIALS: "never"
80+
81+
- name: Build translatable strings
82+
run: tox -e gettext
83+
env:
84+
# We've already built them.
85+
QISKIT_DOCS_BUILD_TUTORIALS: "never"
86+
87+
- name: Store built documentation artifact
88+
uses: actions/upload-artifact@v3
89+
with:
90+
name: qiskit-docs
91+
path: |
92+
./docs/_build/html/*
93+
!**/.doctrees
94+
!**/.buildinfo
95+
if-no-files-found: error
96+
97+
- name: Store translatable strings artifact
98+
uses: actions/upload-artifact@v3
99+
with:
100+
name: qiskit-translatables
101+
path: ./docs/locale/en/*
102+
if-no-files-found: error
103+
104+
deploy:
105+
if: github.event_name != 'workflow_dispatch' || inputs.do_deployment
106+
name: Deploy to qiskit.org
107+
needs: [build]
108+
runs-on: ubuntu-latest
109+
110+
steps:
111+
- uses: actions/checkout@v3
112+
with:
113+
path: qiskit
114+
115+
- uses: actions/download-artifact@v3
116+
with:
117+
name: qiskit-docs
118+
path: deploy
119+
120+
- id: choose
121+
name: Choose deployment location(s)
122+
run: |
123+
set -e
124+
declare -a prefixes
125+
case ${{ github.event_name }} in
126+
push)
127+
case ${{ github.ref_type }} in
128+
branch)
129+
if [[ "$GITHUB_REF_NAME" != "main" ]]; then
130+
echo "Push to unhandled branch '$GITHUB_REF_NAME'" >&2
131+
exit 1
132+
fi
133+
134+
prefixes+=( "dev" )
135+
;;
136+
tag)
137+
tag=$GITHUB_REF_NAME
138+
echo "Full tag: ${tag}"
139+
IFS=. read -ra version <<< "$tag"
140+
minor_version="${version[0]}.${version[1]}"
141+
echo "Minor version: ${minor_version}"
142+
prefixes+=( "stable/${minor_version}" )
143+
if [[ "$tag" == "$LATEST_TAG" ]]; then
144+
# Deploy to the root as well.
145+
prefixes+=( "" )
146+
fi
147+
;;
148+
*)
149+
echo "Unhandled reference type '${{ github.ref_type }}'" >&2
150+
exit 1
151+
;;
152+
esac
153+
;;
154+
workflow_dispatch)
155+
prefixes+=( "$WORKFLOW_DISPATCH_PREFIX" )
156+
;;
157+
*)
158+
echo "Unhandled GitHub event ${{ github.event_name }}" >&2
159+
exit 1
160+
;;
161+
esac
162+
# Join the array of prefixes into a colon-delimited list for
163+
# serialisation. This includes a trailing colon, so we can detect
164+
# the presence of the empty string, even if it's the only prefix.
165+
if [[ "${#prefixes[@]}" -gt 0 ]]; then
166+
joined_prefixes=$(printf "%s:" "${prefixes[@]}")
167+
echo "Chosen deployment prefixes: '$joined_prefixes'"
168+
echo "joined_prefixes=$joined_prefixes" >> "$GITHUB_OUTPUT"
169+
else
170+
echo "Nothing to deploy to."
171+
fi
172+
env:
173+
LATEST_TAG: ${{ needs.build.outputs.latest_tag }}
174+
GITHUB_REF_NAME: ${{ github.ref_name }}
175+
WORKFLOW_DISPATCH_PREFIX: ${{ inputs.deploy_prefix }}
176+
177+
- name: Install rclone
178+
run: |
179+
set -e
180+
curl https://downloads.rclone.org/rclone-current-linux-amd64.deb -o rclone.deb
181+
sudo apt-get install -y ./rclone.deb
182+
183+
- name: Deploy to qiskit.org
184+
if: ${{ steps.choose.outputs.joined_prefixes != '' }}
185+
run: |
186+
set -e
187+
RCLONE_CONFIG=$(rclone config file | tail -1)
188+
openssl aes-256-cbc -K "$RCLONE_KEY" -iv "$RCLONE_IV" -in qiskit/tools/rclone.conf.enc -out "$RCLONE_CONFIG" -d
189+
IFS=: read -ra prefixes <<< "$JOINED_PREFIXES"
190+
for prefix in "${prefixes[@]}"; do
191+
# The 'documentation' bit of the prefix is hard-coded in this step
192+
# rather than being chosen during the prefix-choosing portion
193+
# because we don't want to allow the 'workflow_dispatch' event
194+
# trigger to accidentally allow a deployment to a dodgy prefix that
195+
# wipes out _everything_ on qiskit.org.
196+
location=documentation/$prefix
197+
echo "Deploying to 'qiskit.org/$location'"
198+
rclone sync --progress --exclude-from qiskit/tools/docs_exclude.txt deploy "IBMCOS:qiskit-org-web-resources/$location"
199+
done
200+
env:
201+
JOINED_PREFIXES: ${{ steps.choose.outputs.joined_prefixes }}
202+
RCLONE_KEY: ${{ secrets.ENCRYPTED_RCLONE_KEY}}
203+
RCLONE_IV: ${{ secrets.ENCRYPTED_RCLONE_IV }}
204+
205+
deploy_translatables:
206+
if: (github.event_name == 'workflow_dispatch' && inputs.do_translatables) || (github.event_name == 'push' && github.ref_type == 'tag' && github.ref_name == needs.build.outputs.latest_tag)
207+
name: Push translatable strings
208+
needs: [build]
209+
runs-on: ubuntu-latest
210+
211+
steps:
212+
- uses: actions/checkout@v3
213+
with:
214+
path: 'qiskit'
215+
216+
- uses: actions/download-artifact@v3
217+
with:
218+
name: qiskit-translatables
219+
path: 'deploy'
220+
221+
- name: Decrypt SSH secret key
222+
id: ssh_key
223+
run: |
224+
set -e
225+
ssh_key=$(openssl enc -aes-256-cbc -d -in qiskit/tools/github_poBranch_update_key.enc -K $SSH_UPDATE_KEY -iv $SSH_UPDATE_IV)
226+
echo "::add-mask::${ssh_key}"
227+
echo "ssh_key=${ssh_key}" >> "$GITHUB_OUTPUT"
228+
env:
229+
SSH_UPDATE_KEY: ${{ secrets.ENCRYPTED_DEPLOY_PO_BRANCH_KEY }}
230+
SSH_UPDATE_IV: ${{ secrets.ENCRYPTED_DEPLOY_PO_BRANCH_IV }}
231+
232+
- uses: actions/checkout@v3
233+
with:
234+
repository: 'qiskit-community/qiskit-translations'
235+
path: 'qiskit-translations'
236+
ssh-key: '${{ steps.ssh_key.outputs.ssh_key }}'
237+
238+
- name: Remove ignored documents
239+
run: rm -r LC_MESSAGES/{apidocs,stubs}
240+
working-directory: 'deploy'
241+
242+
- name: Push changes to translations repository
243+
run: |
244+
set -e
245+
shopt -s failglob
246+
# Bring the new `.po` target files into the repository.
247+
git rm -r --ignore-unmatch docs/locale/en
248+
mv "${{ github.workspace }}/deploy" docs/locale/en
249+
# Update the ways to recreate the build.
250+
cp "${{ github.workspace }}/qiskit/"{setup.py,requirements-*.txt,constraints.txt} .
251+
git add .
252+
253+
cat > COMMIT_MSG << EOF
254+
Automated documentation update to add .po files from ${{ github.repository }}
255+
256+
skip ci
257+
258+
Commit: ${{ github.sha }}
259+
GitHub Actions run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
260+
EOF
261+
262+
git config user.name "Qiskit Autodeploy"
263+
git config user.email "qiskit@qiskit.org"
264+
git commit -F COMMIT_MSG
265+
git push origin
266+
working-directory: 'qiskit-translations'

‎.gitignore‎

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -76,8 +76,6 @@ instance/
7676
# Scrapy stuff:
7777
.scrapy
7878

79-
# Sphinx documentation
80-
docs/_build/
8179

8280
# PyBuilder
8381
target/
@@ -118,10 +116,6 @@ tutorial/rst/_build/*
118116

119117
test/python/test_qasm_python_simulator.pdf
120118

121-
doc/_build/*
122-
123-
doc/**/_autodoc
124-
125119
qiskit/bin/*
126120

127121
test/python/test_save.json
@@ -142,8 +136,11 @@ src/qasm-simulator-cpp/test/qubit_vector_tests
142136
qiskit/transpiler/passes/**/cython/**/*.cpp
143137
qiskit/quantum_info/states/cython/*.cpp
144138

145-
docs/stubs/*
146-
executed_tutorials/
139+
# Sphinx documentation
140+
/docs/_build
141+
/docs/stubs
142+
/docs/locale
143+
/executed_tutorials
147144

148145
# Notebook testing images
149146
test/visual/mpl/circuit/circuit_results/*.png

‎azure-pipelines.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ parameters:
6060
type: string
6161
default: "3.8"
6262

63+
# Sync with 'python-version' in '.github/workflows/docs_deploy.yml'.
6364
- name: "documentationPythonVersion"
6465
displayName: "Version of Python to use to build Sphinx documentation"
6566
type: string

0 commit comments

Comments
 (0)
⚡