Support PE Fastq file splitting

[marchoeppner]

So this has been discussed on gitter, but I didn't see a formal feature request for this yet, so:

Currently, splitting of FastQ files has two limitations I can see:

• output is uncompressed (wasteful)
• method only works on single FastQ files

The main issue to me is the second point - while this can be worked around with tuples and merging, it would be much neater if splitting would work for PE reads natively.

I guess having each chunk compressed with gzip at the same time won't work since this would create too much load on the node running the NF process?

[pditommaso]

Hi Marc, these are both interesting improvements. The compression is relatively easy, the second one a bit more challenging. The only problem is there's already a long to-do list and I'm not able to say when I can work on this. Any contribution is welcome.

[pditommaso]

I've created a separate issue for gzip compression and keep this for PE splitting. See #484.

[pditommaso]

It required almost a week-end of pain but paired-end splitting is now supported. It only requires to add the parameter pe:true when invoking splitFastq operator. If you want to give a try define the variable NXF_VER=0.26.0-beta5 then use NF as usual.

Feedback is welcome.