Skip to content

Commit cf795a9

Browse files
committed
Initial pass at full documentation for Conda-based dependency development.
- Add a new advanced topic for developing tools with Conda dependencies with 4 main sections: 1. Using the planemo conda commands such conda_init, conda_install, conda_env, lint --conda_requirements, and test. 2. Finding existing Conda packages - using the Anaconda site, conda search, or a new "planemo conda_search" command for searching best practice channels. 3. A formal exercise based on the tools-iuc pear.xml tool. 4. A short stub of a section containing resources on building and contributing Conda recipes. - Add a project template for downloading the completed intro seqtk_seq.xml example for testing out planemo conda commands. - Add a project template for a Conda exercise based on the pear tool from tools-iuc. - Add a ``planemo conda_search`` command for searching best practice channels from the command line. I think this can serve as the template and example for the first third of the "Conda and Containers - A Developer Perspective" workshop at the GCC 2017.
1 parent db73773 commit cf795a9

18 files changed

Lines changed: 872 additions & 3 deletions

docs/_writing_dependencies_conda.rst

Lines changed: 318 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Dependencies and Docker
2+
===========================================
3+

docs/_writing_using_seqtk.rst

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,7 @@ install Seqtk - but however you obtain it should be fine.
77

88
::
99

10-
$ conda config --add channels r
11-
$ conda config --add channels bioconda
12-
$ conda install seqtk
10+
$ conda install -c bioconda seqtk=1.2
1311
... seqtk installation ...
1412
$ seqtk seq
1513
Usage: seqtk seq [options] <in.fq>|<in.fa>

docs/commands/conda_search.rst

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
2+
``conda_search`` command
3+
======================================
4+
5+
This section is auto-generated from the help text for the planemo command
6+
``conda_search``. This help message can be generated with ``planemo conda_search
7+
--help``.
8+
9+
**Usage**::
10+
11+
planemo conda_search [OPTIONS] TERM
12+
13+
**Help**
14+
15+
Perform conda search with Planemo's conda.
16+
17+
Implicitly adds channels Planemo is configured with.
18+
19+
**Options**::
20+
21+
22+
--conda_prefix DIRECTORY Conda prefix to use for conda dependency
23+
commands.
24+
--conda_exec PATH Location of conda executable.
25+
--conda_debug Enable more verbose conda logging.
26+
--conda_channels, --conda_ensure_channels TEXT
27+
Ensure conda is configured with specified
28+
comma separated list of channels.
29+
--help Show this message and exit.
30+

docs/planemo.commands.rst

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,14 @@ planemo.commands.cmd_conda_lint module
100100
:undoc-members:
101101
:show-inheritance:
102102

103+
planemo.commands.cmd_conda_search module
104+
----------------------------------------
105+
106+
.. automodule:: planemo.commands.cmd_conda_search
107+
:members:
108+
:undoc-members:
109+
:show-inheritance:
110+
103111
planemo.commands.cmd_config_init module
104112
---------------------------------------
105113

@@ -204,6 +212,14 @@ planemo.commands.cmd_normalize module
204212
:undoc-members:
205213
:show-inheritance:
206214

215+
planemo.commands.cmd_open module
216+
--------------------------------
217+
218+
.. automodule:: planemo.commands.cmd_open
219+
:members:
220+
:undoc-members:
221+
:show-inheritance:
222+
207223
planemo.commands.cmd_profile_create module
208224
------------------------------------------
209225

docs/planemo.reports.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,14 @@ planemo.reports.build_report module
1212
:undoc-members:
1313
:show-inheritance:
1414

15+
planemo.reports.xunit_handler module
16+
------------------------------------
17+
18+
.. automodule:: planemo.reports.xunit_handler
19+
:members:
20+
:undoc-members:
21+
:show-inheritance:
22+
1523

1624
Module contents
1725
---------------

docs/writing_advanced.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ available - check out tutorial_ if you have never developed a Galaxy tool.
1313
.. include:: _writing_macros.rst
1414
.. include:: _writing_tool_metadata.rst
1515
.. include:: _writing_clusters.rst
16+
.. include:: _writing_dependencies_conda.rst
1617

1718
.. _Galaxy: http://galaxyproject.org/
1819
.. _Docker: https://www.docker.com/
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
"""Module describing the planemo ``conda_search`` command."""
2+
from __future__ import print_function
3+
4+
import click
5+
6+
from planemo import options
7+
from planemo.cli import command_function
8+
from planemo.conda import build_conda_context
9+
10+
11+
@click.command('conda_search')
12+
@options.conda_target_options()
13+
@click.argument(
14+
"term",
15+
metavar="TERM",
16+
type=str,
17+
nargs=1,
18+
)
19+
@command_function
20+
def cli(ctx, term, **kwds):
21+
"""Perform conda search with Planemo's conda.
22+
23+
Implicitly adds channels Planemo is configured with.
24+
"""
25+
conda_context = build_conda_context(ctx, use_planemo_shell_exec=False, **kwds)
26+
conda_context.exec_command("search", [term])
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# repository published to https://toolshed.g2.bx.psu.edu/repos/iuc/pear
2+
owner: planemo
3+
name: pear
4+
description: PEAR evaluates all possible paired-end read overlaps
5+
long_description: |
6+
PEAR evaluates all possible paired-end read overlaps and without requiring the target fragment size as input. In addition, it implements a statistical test for
7+
minimizing false-positive results. Together with a highly optimized implementation, it can merge millions of paired end reads within a couple of minutes on a
8+
standard desktop computer. Repository-Maintainer: Björn Grüning
9+
remote_repository_url: https://github.com/galaxyproject/planemo/tree/master/project_templates/conda_exercise/exercise_1
10+
Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
<tool id="iuc_pear" name="Pear" version="0.9.6.1">
2+
<description>Paired-End read merger</description>
3+
<stdio>
4+
<exit_code range="1:" />
5+
<exit_code range=":-1" />
6+
<regex match="Error:" />
7+
<regex match="Exception:" />
8+
</stdio>
9+
<command>
10+
<![CDATA[
11+
pear
12+
#if str( $library.type ) == "paired":
13+
-f "$library.forward"
14+
-r "$library.reverse"
15+
#if $library.forward.is_of_type( 'fastqillumina' ):
16+
--phred-base 64
17+
#else:
18+
--phred-base 33
19+
#end if
20+
#else
21+
## prepare collection
22+
-f "$library.input_collection.forward"
23+
-r "$library.input_collection.reverse"
24+
#if $library.input_collection.forward.is_of_type( 'fastqillumina' ):
25+
--phred-base 64
26+
#else:
27+
--phred-base 33
28+
#end if
29+
#end if
30+
31+
--output pear
32+
--p-value $pvalue
33+
--min-overlap $min_overlap
34+
#if int($max_assembly_length) > 0:
35+
--max-asm-length $max_assembly_length
36+
#end if
37+
--min-asm-length $min_assembly_length
38+
--min-trim-length $min_trim_length
39+
--quality-theshold $quality_threshold
40+
--max-uncalled-base $max_uncalled_base
41+
--test-method $test_method
42+
--empirical-freqs $empirical_freqs
43+
-j "\${GALAXY_SLOTS:-8}"
44+
--score-method $score_method
45+
--cap $cap
46+
$nbase
47+
]]>
48+
</command>
49+
<inputs>
50+
<conditional name="library">
51+
<param name="type" type="select" label="Dataset type">
52+
<option value="paired">Paired-end</option>
53+
<option value="paired_collection">Paired-end Dataset Collection</option>
54+
</param>
55+
<when value="paired">
56+
<param name="forward" type="data" format="fastqillumina,fastqsanger"
57+
label="Name of file that contains the forward paired-end reads" help="-f" />
58+
<param name="reverse" type="data" format="fastqillumina,fastqsanger"
59+
label="Name of file that contains the reverse paired-end reads" help="-r" />
60+
</when>
61+
<when value="paired_collection">
62+
<param name="input_collection" format="fastqillumina,fastqsanger"
63+
type="data_collection" collection_type="paired"
64+
label="FASTQ Paired Dataset" help="Nucleotide-space: Must have PHRED-scaled quality values. (-f and -r)" />
65+
</when>
66+
</conditional>
67+
68+
<!-- optional -->
69+
<param name="pvalue" type="float" value="0.01" min="0" optional="True" max="1" label="Specify a p-value for the statistical test"
70+
help="If the computed p-value of a possible assembly exceeds the specified p-value then the paired-end read will not be assembled. Setting 1.0 disables the test. (--p-value)" />
71+
72+
<param name="min_overlap" type="integer" value="10" optional="True" label="Minimum overlap size"
73+
help="The minimum overlap may be set to 1 when the statistical test is used. However, further restricting the minimum overlap size to a proper value may reduce false-positive assembles. (--min-overlap)" />
74+
75+
<param name="max_assembly_length" type="integer" value="0" optional="True" label="Maximum possible length of the assembled sequences"
76+
help="Setting this value to 0 disables the restriction and assembled sequences may be arbitrary long. (--max-assembly-length)" />
77+
78+
<param name="min_assembly_length" type="integer" value="50" optional="True" label="Minimum possible length of the assembled sequences"
79+
help="Setting this value to 0 disables the restriction and assembled sequences may be arbitrary short. (--min-assembly-length)" />
80+
81+
<param name="min_trim_length" type="integer" value="1" optional="True" label="Minimum length of reads after trimming the low quality part"
82+
help="See option -q. (--min-trim-length)" />
83+
84+
<param name="quality_threshold" type="integer" value="0" optional="True" label="Quality score threshold for trimming the low quality part of a read"
85+
help="If the quality scores of two consecutive bases are strictly less than the specified threshold, the rest of the read will be trimmed. (--quality-threshold)" />
86+
87+
<param name="max_uncalled_base" type="float" value="1.0" min="0" optional="True" max="1" label="Maximal proportion of uncalled bases in a read"
88+
help="Setting this value to 0 will cause PEAR to discard all reads containing uncalled bases. The other extreme setting is 1 which causes PEAR to process all reads independent on the number of uncalled bases. (--max-uncalled-base)" />
89+
90+
<param name="cap" type="integer" value="40" optional="True" label="Specify the upper bound for the resulting quality score"
91+
help="If set to zero, capping is disabled. (--cap)" />
92+
93+
<param name="test_method" type="select" label="Type of statistical test" help="(--test-method)">
94+
<option value="1" selected="True">Given the minimum allowed overlap, test using the highest OES (1)</option>
95+
<option value="2">Use the acceptance probability (2)</option>
96+
</param>
97+
98+
<param name="empirical_freqs" type="boolean" truevalue="-e" falsevalue="" checked="false"
99+
label="Disable empirical base frequencies" help="(--empirical-freqs)" />
100+
<param name="nbase" type="boolean" truevalue="--nbase" falsevalue="" checked="false"
101+
label="Use N base if uncertain" help="When merging a base-pair that consists of two non-equal bases out of which none is degenerate, set the merged base to N and use the highest quality score of the two bases. (--nbase)" />
102+
103+
<param name="score_method" type="select" label="Scoring method" help="(--score-method)">
104+
<option value="1">OES with +1 for match and -1 for mismatch</option>
105+
<option value="2" selected="True">Assembly score (AS) use +1 for match and -1 for mismatch multiplied by base quality scores</option>
106+
<option value="3">Ignore quality scores and use +1 for a match and -1 for a mismatch</option>
107+
</param>
108+
109+
<param name="outputs" type="select" display="checkboxes" optional="False" multiple="True" label="Output files">
110+
<option value="assembled" selected="True">Assembled reads</option>
111+
<option value="forward">Forward unassembled reads</option>
112+
<option value="reverse">Reverse unassembled reads</option>
113+
<option value="discarded">Discarded reads</option>
114+
</param>
115+
</inputs>
116+
<outputs>
117+
<data format="input" name="assembled_reads" from_work_dir="pear.assembled.fastq" label="${tool.name} on ${on_string}: Assembled reads">
118+
<filter>'assembled' in outputs</filter>
119+
</data>
120+
<data format="input" name="unassembled_forward_reads" from_work_dir="pear.unassembled.forward.fastq" label="${tool.name} on ${on_string}: Unassembled forward reads">
121+
<filter>'forward' in outputs</filter>
122+
</data>
123+
<data format="input" name="unassembled_reverse_reads" from_work_dir="pear.unassembled.reverse.fastq" label="${tool.name} on ${on_string}: Unassembled reverse reads">
124+
<filter>'reverse' in outputs</filter>
125+
</data>
126+
<data format="input" name="discarded_reads" from_work_dir="pear.discarded.fastq" label="${tool.name} on ${on_string}: Discarded reads">
127+
<filter>'discarded' in outputs</filter>
128+
</data>
129+
</outputs>
130+
<tests>
131+
<test>
132+
<param name="forward" value="forward.fastq" ftype="fastqsanger" />
133+
<param name="reverse" value="reverse.fastq" ftype="fastqsanger" />
134+
<param name="min_overlap" value="10" />
135+
<param name="min_assembly_length" value="50" />
136+
<param name="cap" value="0" />
137+
<param name="outputs" value="assembled,forward" />
138+
<output name="assembled_reads" file="pear_assembled_results1.fastq" ftype="fastqsanger"/>
139+
<output name="unassembled_forward_reads" file="pear_unassembled_forward_results1.fastq" ftype="fastqsanger"/>
140+
</test>
141+
</tests>
142+
<help>
143+
<![CDATA[
144+
145+
**What it does**
146+
147+
PEAR_ is an ultrafast, memory-efficient and highly accurate pair-end read merger.
148+
PEAR evaluates all possible paired-end read overlaps and without requiring the target fragment
149+
size as input. In addition, it implements a statistical test for minimizing false-positive results.
150+
Together with a highly optimized implementation, it can merge millions of paired end reads within a couple of minutes
151+
on a standard desktop computer.
152+
153+
For more information please look at the documentation_ and `github repository`_.
154+
155+
.. _PEAR: http://sco.h-its.org/exelixis/web/software/pear/
156+
.. _documentation: http://sco.h-its.org/exelixis/web/software/pear/doc.html
157+
.. _github repository: https://github.com/xflouris/PEAR
158+
159+
160+
]]>
161+
</help>
162+
<citations>
163+
<citation type="doi">10.1093/bioinformatics/btt593</citation>
164+
</citations>
165+
</tool>

0 commit comments

Comments
 (0)