Skip to content

Commit b28e037

Browse files
jmchiltonjmchilton
committed
Documentation improvements, rebuild docs and expand workflow test docs.
1 parent 23b26f1 commit b28e037

10 files changed

Lines changed: 168 additions & 3 deletions

‎docs/commands/run.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ Planemo command for running tools and jobs.
144144
Where to store outputs of a 'run' task.
145145
--output_json PATH Where to store JSON dictionary describing
146146
outputs of a 'run' task.
147-
--engine [galaxy|docker_galaxy|cwltool|external_galaxy]
147+
--engine [galaxy|docker_galaxy|cwltool|toil|external_galaxy]
148148
Select an engine to run or test aritfacts such
149149
as tools and workflows. Defaults to a local
150150
Galaxy, but running Galaxy within a Docker

‎docs/commands/test.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@ please careful and do not try this against production Galaxy instances.
175175
Summary style printed to planemo's standard
176176
output (see output reports for more complete
177177
summary). Set to 'none' to disable completely.
178-
--engine [galaxy|docker_galaxy|cwltool|external_galaxy]
178+
--engine [galaxy|docker_galaxy|cwltool|toil|external_galaxy]
179179
Select an engine to run or test aritfacts such
180180
as tools and workflows. Defaults to a local
181181
Galaxy, but running Galaxy within a Docker

‎docs/example_travis_gxwf_docker.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
language: python
2+
sudo: true
3+
python: 2.7
4+
env:
5+
global:
6+
- PLANEMO_TEST_SCRIPT=https://raw.githubusercontent.com/galaxyproject/planemo/master/scripts/run_galaxy_workflow_tests.sh
7+
- PLANEMO_TEST_STYLE=docker_serve_and_test
8+
- PLANEMO_TARGET="planemo==0.50.1"
9+
- PLANEMO_DOCKER_GALAXY_IMAGE="quay.io/bgruening/galaxy:18.01"
10+
matrix:
11+
- WORKFLOW_TEST=example1/ref-rnaseq.ga
12+
- WORKFLOW_TEST=example2/chipseq.ga
13+
14+
script: bash <(curl -s "$PLANEMO_TEST_SCRIPT") "$WORKFLOW_TEST"
15+
16+
services:
17+
- docker

‎docs/example_travis_gxwf_native.yml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
language: python
2+
python: 2.7
3+
env:
4+
global:
5+
- PLANEMO_TEST_SCRIPT=https://raw.githubusercontent.com/galaxyproject/planemo/master/scripts/run_galaxy_workflow_tests.sh
6+
- PLANEMO_TEST_STYLE=serve_and_test
7+
- PLANEMO_TARGET="planemo==0.50.1"
8+
- PLANEMO_GALAXY_BRANCH="release_18.05"
9+
matrix:
10+
- WORKFLOW_TEST=example1/ref-rnaseq.ga
11+
- WORKFLOW_TEST=example2/chipseq.ga
12+
13+
script: bash <(curl -s "$PLANEMO_TEST_SCRIPT") "$WORKFLOW_TEST"
14+
15+
services:
16+
- postgres

‎docs/planemo.cwl.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,14 @@ planemo.cwl.script module
2020
:undoc-members:
2121
:show-inheritance:
2222

23+
planemo.cwl.toil module
24+
-----------------------
25+
26+
.. automodule:: planemo.cwl.toil
27+
:members:
28+
:undoc-members:
29+
:show-inheritance:
30+
2331

2432
Module contents
2533
---------------

‎docs/planemo.engine.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,14 @@ planemo.engine.interface module
3636
:undoc-members:
3737
:show-inheritance:
3838

39+
planemo.engine.toil module
40+
--------------------------
41+
42+
.. automodule:: planemo.engine.toil
43+
:members:
44+
:undoc-members:
45+
:show-inheritance:
46+
3947

4048
Module contents
4149
---------------

‎docs/planemo.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,14 @@ planemo.config module
8787
:undoc-members:
8888
:show-inheritance:
8989

90+
planemo.deps module
91+
-------------------
92+
93+
.. automodule:: planemo.deps
94+
:members:
95+
:undoc-members:
96+
:show-inheritance:
97+
9098
planemo.docker module
9199
---------------------
92100

‎docs/run_galaxy_workflow_tests.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../scripts/run_galaxy_workflow_tests.sh

‎docs/test_format.rst‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -247,10 +247,32 @@ WIP - stay tuned - but this will allow testing workflows against running Galaxy
247247
Galaxy Testing Template
248248
-------------------------
249249

250-
WIP - https://github.com/jmchilton/planemo-workflow-test-template.
250+
The following a script that can be used with `continuous integration`_ (CI) services such
251+
Travis_ to test Galaxy workflows in a Github repository. This shell script can be configured via
252+
various environment variables and shows off some of the modalities Planemo ``test`` should work in
253+
(there may be bugs but we are trying to stablize this functionality).
251254

255+
.. literalinclude:: run_galaxy_workflow_tests.sh
256+
:language: bash
257+
258+
A Travis_ configuration file (.travis.yml) that would test workflows using a Docker Galaxy image might
259+
look like:
260+
261+
.. literalinclude:: example_travis_gxwf_docker.yml
262+
:language: bash
263+
264+
To skip Docker and instead test with a native Galaxy instance and postgres database one might use the
265+
configuration:
266+
267+
.. literalinclude:: example_travis_gxwf_native.yml
268+
:language: bash
269+
270+
.. _Travis: https://travis-ci.org/
271+
.. _test command: http://planemo.readthedocs.org/en/latest/commands.html#test-command
272+
.. _run command: http://planemo.readthedocs.org/en/latest/commands.html#run-command
252273
.. _SHA1: https://en.wikipedia.org/wiki/SHA-1
253274
.. _checksum: https://en.wikipedia.org/wiki/Checksum
275+
.. _continuous integration: https://en.wikipedia.org/wiki/Continuous_integration
254276
.. _Common Workflow Language: https://www.commonwl.org/
255277
.. _CWL: https://www.commonwl.org/
256278
.. _cwltool: https://github.com/common-workflow-language/cwltool

‎scripts/run_galaxy_workflow_tests.sh‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
#!/bin/bash
2+
3+
: ${PLANEMO_TARGET:="planemo==0.50.1"}
4+
: ${PLANEMO_OPTIONS:=""} # e.g. PLANEMO_OPTIONS="--verbose"
5+
: ${PLANEMO_PROFILE_NAME:="wxflowtest"}
6+
: ${PLANEMO_SERVE_PORT:="9019"}
7+
: ${PLANEMO_GALAXY_BRANCH:="master"}
8+
: ${PLANEMO_TEST_STYLE:="serve_and_test"} # profile_serve_and_test, serve_and_test, docker_serve_and_test, test, docker_test
9+
: ${PLANEMO_SERVE_DATABASE_TYPE:="postgres"} # used if not using Docker with PLANEMO_TEST_STYLE
10+
: ${PLANEMO_DOCKER_GALAXY_IMAGE:="quay.io/bgruening/galaxy:18.01"} # used if used Docker with PLANEMO_TEST_STYLE
11+
12+
GALAXY_URL="http://localhost:$PLANEMO_SERVE_PORT"
13+
14+
# Ensure Planemo is installed.
15+
if [ ! -d .venv ]; then
16+
virtualenv .venv
17+
. .venv/bin/activate
18+
pip install -U pip
19+
# Intentionally expand wildcards in PLANEMO_TARGET.
20+
shopt -s extglob
21+
pip install ${PLANEMO_TARGET}
22+
fi
23+
. .venv/bin/activate
24+
25+
# Run test.
26+
# This example shows off a bunch of different ways one could test with Planemo,
27+
# but for actual workflow testing projects - probably best just to take one of the last
28+
# two very easy invocations to simplify things.
29+
if [ "$PLANEMO_TEST_STYLE" = "profile_serve_and_test" ]; then
30+
planemo $PLANEMO_OPTIONS profile_create \
31+
--database_type "$PLANEMO_SERVE_DATABASE_TYPE" \
32+
"$PLANEMO_PROFILE_NAME"
33+
planemo $PLANEMO_OPTIONS serve \
34+
--daemon \
35+
--galaxy_branch "$PLANEMO_GALAXY_BRANCH" \
36+
--profile "$PLANEMO_PROFILE_NAME" \
37+
--port "$PLANEMO_SERVE_PORT" \
38+
"$1"
39+
planemo $PLANEMO_OPTIONS test \
40+
--galaxy_url "$GALAXY_URL" \
41+
--engine external_galaxy \
42+
"$1"
43+
elif [ "$PLANEMO_TEST_STYLE" = "serve_and_test" ]; then
44+
planemo $PLANEMO_OPTIONS serve \
45+
--daemon \
46+
--galaxy_branch "$PLANEMO_GALAXY_BRANCH" \
47+
--database_type "$PLANEMO_SERVE_DATABASE_TYPE" \
48+
--port "$PLANEMO_SERVE_PORT" \
49+
"$1"
50+
planemo $PLANEMO_OPTIONS test \
51+
--galaxy_url "$GALAXY_URL" \
52+
--engine external_galaxy \
53+
"$1"
54+
elif [ "$PLANEMO_TEST_STYLE" = "docker_serve_and_test" ]; then
55+
docker pull "${PLANEMO_DOCKER_GALAXY_IMAGE}"
56+
planemo $PLANEMO_OPTIONS serve \
57+
--daemon \
58+
--engine docker_galaxy \
59+
--docker_galaxy_image "${PLANEMO_DOCKER_GALAXY_IMAGE}" \
60+
--docker_extra_volume . \
61+
--port "$PLANEMO_SERVE_PORT" \
62+
"$1"
63+
planemo $PLANEMO_OPTIONS test \
64+
--galaxy_url "$GALAXY_URL" \
65+
--engine external_galaxy \
66+
"$1"
67+
elif [ "$PLANEMO_TEST_STYLE" = "test" ]; then
68+
# TODO: This variant is broken in initial tests for some reason.
69+
planemo $PLANEMO_OPTIONS test \
70+
--database_type "$PLANEMO_SERVE_DATABASE_TYPE" \
71+
--galaxy_branch "$PLANEMO_GALAXY_BRANCH" \
72+
"$1"
73+
elif [ "$PLANEMO_TEST_STYLE" = "docker_test" ]; then
74+
# TODO: This variant isn't super usable yet because there is too much logging, hence the dev null
75+
# redirect.
76+
docker pull "${PLANEMO_DOCKER_GALAXY_IMAGE}"
77+
planemo $PLANEMO_OPTIONS test \
78+
--engine docker_galaxy \
79+
--docker_galaxy_image "${PLANEMO_DOCKER_GALAXY_IMAGE}" \
80+
--docker_extra_volume . \
81+
"$1" > /dev/null
82+
else
83+
echo "Unknown test style ${PLANEMO_TEST_STYLE}"
84+
exit 1
85+
fi

0 commit comments

Comments
 (0)
⚡