Skip to content

Commit ee2d9de

Browse files
jmchiltonjmchilton
committed
Enhancements to Galaxy profiles and workflow testing.
Add a new profile database type option "postgres_docker". The existing "postgres" type option expects a local database to be accessible via the "psql" command on Planemo's PATH - probably few people have postgres running directly on their development machine and fewer still have it configured to allow passwordless entry. This newer variant is a much simpler interface - it starts a named container for the official Docker postgres image and uses that to manage the profile. This should make using profiles easier for most developers. This new modality does have some shortcomings - it should probably be redone to persist the database somewhere -or- start and stop the same container (currently when the container is stopped the data is gone). This also updates the workflow test that encounters Galaxy locks to pickup the environment variable PLANEMO_TEST_WORKFLOW_RUN_PROFILE and PLANEMO_TEST_WORKFLOW_RUN_PROFILE_DATABASE_TYPE and use these to create and use a profile during testing if set as well as setting that variable in Travis to leverage the postgres service already configured for other tests. This more less fixes what I had in mind with #772 but it would be nice to leverage galaxyproject/galaxy#4887 for greater test isolation.
1 parent dedeb65 commit ee2d9de

16 files changed

Lines changed: 279 additions & 65 deletions

‎.travis.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ matrix:
2222
- env: TOX_ENV=py34
2323
- env: TOX_ENV=py27-lint-docstrings
2424

25-
script: PLANEMO_ENABLE_POSTGRES_TESTS=1 PLANEMO_SKIP_CWLTOOL_TESTS=1 tox -e $TOX_ENV
25+
script: PLANEMO_ENABLE_POSTGRES_TESTS=1 PLANEMO_SKIP_CWLTOOL_TESTS=1 PLANEMO_TEST_WORKFLOW_RUN_PROFILE=travisworkflowtests tox -e $TOX_ENV
2626

2727
after_success:
2828
- coveralls

‎docs/commands/profile_create.rst‎

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,17 @@ Create a profile.
1717

1818

1919
--postgres Use postgres database type.
20-
--database_type [postgres|sqlite]
21-
Type of database to use for profile -
22-
currently only 'postgres' is available.
20+
--database_type [postgres|postgres_docker|sqlite|auto]
21+
Type of database to use for profile - 'auto',
22+
'sqlite', 'postgres', and 'postgres_docker'
23+
are available options. Use postgres to use an
24+
existing postgres server you user can access
25+
without a password via the psql command. Use
26+
postgres_docker to have Planemo manage a
27+
docker container running postgres. Data with
28+
postgres_docker is not yet persisted past when
29+
you restart the docker container launched by
30+
Planemo so be careful with this option.
2331
--postgres_psql_path TEXT Name or or path to postgres client binary
2432
(psql).
2533
--postgres_database_user TEXT Postgres username for managed development
@@ -33,5 +41,13 @@ Create a profile.
3341
tools and workflows. Defaults to a local
3442
Galaxy, but running Galaxy within a Docker
3543
container.
44+
--docker_cmd TEXT Command used to launch docker (defaults to
45+
docker).
46+
--docker_sudo / --no_docker_sudo
47+
Flag to use sudo when running docker.
48+
--docker_host TEXT Docker host to target when executing docker
49+
commands (defaults to localhost).
50+
--docker_sudo_cmd TEXT sudo command to use when --docker_sudo is
51+
enabled (defaults to sudo).
3652
--help Show this message and exit.
3753

‎docs/commands/profile_delete.rst‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,17 @@ Delete a profile.
1717

1818

1919
--postgres Use postgres database type.
20-
--database_type [postgres|sqlite]
21-
Type of database to use for profile -
22-
currently only 'postgres' is available.
20+
--database_type [postgres|postgres_docker|sqlite|auto]
21+
Type of database to use for profile - 'auto',
22+
'sqlite', 'postgres', and 'postgres_docker'
23+
are available options. Use postgres to use an
24+
existing postgres server you user can access
25+
without a password via the psql command. Use
26+
postgres_docker to have Planemo manage a
27+
docker container running postgres. Data with
28+
postgres_docker is not yet persisted past when
29+
you restart the docker container launched by
30+
Planemo so be careful with this option.
2331
--postgres_psql_path TEXT Name or or path to postgres client binary
2432
(psql).
2533
--postgres_database_user TEXT Postgres username for managed development

‎docs/commands/run.rst‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -103,12 +103,21 @@ Planemo command for running tools and jobs.
103103
Conda dependency resolution for Galaxy will
104104
auto install conda itself using miniconda if
105105
not availabe on conda_prefix.
106-
--profile TEXT Location of pid file is executed with
107-
--daemon.
106+
--profile TEXT Name of profile (created with the
107+
profile_create command) to use with this
108+
command.
108109
--postgres Use postgres database type.
109-
--database_type [postgres|sqlite]
110-
Type of database to use for profile -
111-
currently only 'postgres' is available.
110+
--database_type [postgres|postgres_docker|sqlite|auto]
111+
Type of database to use for profile - 'auto',
112+
'sqlite', 'postgres', and 'postgres_docker'
113+
are available options. Use postgres to use an
114+
existing postgres server you user can access
115+
without a password via the psql command. Use
116+
postgres_docker to have Planemo manage a
117+
docker container running postgres. Data with
118+
postgres_docker is not yet persisted past when
119+
you restart the docker container launched by
120+
Planemo so be careful with this option.
112121
--postgres_psql_path TEXT Name or or path to postgres client binary
113122
(psql).
114123
--postgres_database_user TEXT Postgres username for managed development

‎docs/commands/serve.rst‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -129,12 +129,21 @@ Galaxy instance.
129129
Conda dependency resolution for Galaxy will
130130
auto install conda itself using miniconda if
131131
not availabe on conda_prefix.
132-
--profile TEXT Location of pid file is executed with
133-
--daemon.
132+
--profile TEXT Name of profile (created with the
133+
profile_create command) to use with this
134+
command.
134135
--postgres Use postgres database type.
135-
--database_type [postgres|sqlite]
136-
Type of database to use for profile -
137-
currently only 'postgres' is available.
136+
--database_type [postgres|postgres_docker|sqlite|auto]
137+
Type of database to use for profile - 'auto',
138+
'sqlite', 'postgres', and 'postgres_docker'
139+
are available options. Use postgres to use an
140+
existing postgres server you user can access
141+
without a password via the psql command. Use
142+
postgres_docker to have Planemo manage a
143+
docker container running postgres. Data with
144+
postgres_docker is not yet persisted past when
145+
you restart the docker container launched by
146+
Planemo so be careful with this option.
138147
--postgres_psql_path TEXT Name or or path to postgres client binary
139148
(psql).
140149
--postgres_database_user TEXT Postgres username for managed development
@@ -149,7 +158,7 @@ Galaxy instance.
149158
--shed_tool_conf TEXT Location of shed tools conf file for Galaxy.
150159
--shed_tool_path TEXT Location of shed tools directory for Galaxy.
151160
--daemon Serve Galaxy process as a daemon.
152-
--pid_file TEXT Location of pid file is executed with
161+
--pid_file PATH Location of pid file is executed with
153162
--daemon.
154163
--cwl Configure Galaxy for use with CWL tool. (this
155164
option is experimental and will be replaced

‎docs/commands/shed_serve.rst‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -127,12 +127,21 @@ logged into and explored interactively.
127127
Conda dependency resolution for Galaxy will
128128
auto install conda itself using miniconda if
129129
not availabe on conda_prefix.
130-
--profile TEXT Location of pid file is executed with
131-
--daemon.
130+
--profile TEXT Name of profile (created with the
131+
profile_create command) to use with this
132+
command.
132133
--postgres Use postgres database type.
133-
--database_type [postgres|sqlite]
134-
Type of database to use for profile -
135-
currently only 'postgres' is available.
134+
--database_type [postgres|postgres_docker|sqlite|auto]
135+
Type of database to use for profile - 'auto',
136+
'sqlite', 'postgres', and 'postgres_docker'
137+
are available options. Use postgres to use an
138+
existing postgres server you user can access
139+
without a password via the psql command. Use
140+
postgres_docker to have Planemo manage a
141+
docker container running postgres. Data with
142+
postgres_docker is not yet persisted past when
143+
you restart the docker container launched by
144+
Planemo so be careful with this option.
136145
--postgres_psql_path TEXT Name or or path to postgres client binary
137146
(psql).
138147
--postgres_database_user TEXT Postgres username for managed development
@@ -146,7 +155,7 @@ logged into and explored interactively.
146155
--database_connection TEXT Database connection string to use for Galaxy.
147156
--shed_tool_conf TEXT Location of shed tools conf file for Galaxy.
148157
--shed_tool_path TEXT Location of shed tools directory for Galaxy.
149-
--pid_file TEXT Location of pid file is executed with
158+
--pid_file PATH Location of pid file is executed with
150159
--daemon.
151160
--skip_dependencies Do not install shed dependencies as part of
152161
repository installation.

‎docs/commands/test.rst‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -123,12 +123,21 @@ please careful and do not try this against production Galaxy instances.
123123
Conda dependency resolution for Galaxy will
124124
auto install conda itself using miniconda if
125125
not availabe on conda_prefix.
126-
--profile TEXT Location of pid file is executed with
127-
--daemon.
126+
--profile TEXT Name of profile (created with the
127+
profile_create command) to use with this
128+
command.
128129
--postgres Use postgres database type.
129-
--database_type [postgres|sqlite]
130-
Type of database to use for profile -
131-
currently only 'postgres' is available.
130+
--database_type [postgres|postgres_docker|sqlite|auto]
131+
Type of database to use for profile - 'auto',
132+
'sqlite', 'postgres', and 'postgres_docker'
133+
are available options. Use postgres to use an
134+
existing postgres server you user can access
135+
without a password via the psql command. Use
136+
postgres_docker to have Planemo manage a
137+
docker container running postgres. Data with
138+
postgres_docker is not yet persisted past when
139+
you restart the docker container launched by
140+
Planemo so be careful with this option.
132141
--postgres_psql_path TEXT Name or or path to postgres client binary
133142
(psql).
134143
--postgres_database_user TEXT Postgres username for managed development

‎docs/commands/tool_factory.rst‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -114,12 +114,21 @@ http://www.ncbi.nlm.nih.gov/pubmed/23024011.
114114
Conda dependency resolution for Galaxy will
115115
auto install conda itself using miniconda if
116116
not availabe on conda_prefix.
117-
--profile TEXT Location of pid file is executed with
118-
--daemon.
117+
--profile TEXT Name of profile (created with the
118+
profile_create command) to use with this
119+
command.
119120
--postgres Use postgres database type.
120-
--database_type [postgres|sqlite]
121-
Type of database to use for profile -
122-
currently only 'postgres' is available.
121+
--database_type [postgres|postgres_docker|sqlite|auto]
122+
Type of database to use for profile - 'auto',
123+
'sqlite', 'postgres', and 'postgres_docker'
124+
are available options. Use postgres to use an
125+
existing postgres server you user can access
126+
without a password via the psql command. Use
127+
postgres_docker to have Planemo manage a
128+
docker container running postgres. Data with
129+
postgres_docker is not yet persisted past when
130+
you restart the docker container launched by
131+
Planemo so be careful with this option.
123132
--postgres_psql_path TEXT Name or or path to postgres client binary
124133
(psql).
125134
--postgres_database_user TEXT Postgres username for managed development
@@ -134,7 +143,7 @@ http://www.ncbi.nlm.nih.gov/pubmed/23024011.
134143
--shed_tool_conf TEXT Location of shed tools conf file for Galaxy.
135144
--shed_tool_path TEXT Location of shed tools directory for Galaxy.
136145
--daemon Serve Galaxy process as a daemon.
137-
--pid_file TEXT Location of pid file is executed with
146+
--pid_file PATH Location of pid file is executed with
138147
--daemon.
139148
--help Show this message and exit.
140149

‎docs/planemo.database.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,14 @@ planemo\.database\.postgres module
2828
:undoc-members:
2929
:show-inheritance:
3030

31+
planemo\.database\.postgres\_docker module
32+
------------------------------------------
33+
34+
.. automodule:: planemo.database.postgres_docker
35+
:members:
36+
:undoc-members:
37+
:show-inheritance:
38+
3139

3240
Module contents
3341
---------------

‎planemo/commands/cmd_profile_create.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
@options.profile_name_argument()
1313
@options.profile_database_options()
1414
@options.serve_engine_option()
15+
@options.docker_config_options()
1516
@command_function
1617
def cli(ctx, profile_name, **kwds):
1718
"""Create a profile."""

0 commit comments

Comments
 (0)
⚡