Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fix number of cpus for modules with piped tools #499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JoseEspinosa merged 2 commits into from
May 18, 2021
Merged

Fix number of cpus for modules with piped tools #499

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b2b1b1a
Split CPUs for piped commands
JoseEspinosa May 17, 2021
a370536
Fix tests, bams no md5 check
JoseEspinosa May 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions software/bowtie/align/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,22 +29,23 @@ process BOWTIE_ALIGN {
tuple val(meta), path('*fastq.gz'), optional:true, emit: fastq

script:
def split_cpus = Math.floor(task.cpus/2)
def software = getSoftwareName(task.process)
def prefix = options.suffix ? "${meta.id}${options.suffix}" : "${meta.id}"
def unaligned = params.save_unaligned ? "--un ${prefix}.unmapped.fastq" : ''
def endedness = meta.single_end ? "$reads" : "-1 ${reads[0]} -2 ${reads[1]}"
"""
INDEX=`find -L ./ -name "*.3.ebwt" | sed 's/.3.ebwt//'`
bowtie \\
--threads $task.cpus \\
--threads ${split_cpus} \\
--sam \\
-x \$INDEX \\
-q \\
$unaligned \\
$options.args \\
$endedness \\
2> ${prefix}.out \\
| samtools view $options.args2 -@ $task.cpus -bS -o ${prefix}.bam -
| samtools view $options.args2 -@ ${split_cpus} -bS -o ${prefix}.bam -

if [ -f ${prefix}.unmapped.fastq ]; then
gzip ${prefix}.unmapped.fastq
Expand Down
13 changes: 7 additions & 6 deletions software/bowtie2/align/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,20 +29,21 @@ process BOWTIE2_ALIGN {
tuple val(meta), path('*fastq.gz'), optional:true, emit: fastq

script:
def software = getSoftwareName(task.process)
def prefix = options.suffix ? "${meta.id}${options.suffix}" : "${meta.id}"
def split_cpus = Math.floor(task.cpus/2)
def software = getSoftwareName(task.process)
def prefix = options.suffix ? "${meta.id}${options.suffix}" : "${meta.id}"
if (meta.single_end) {
def unaligned = params.save_unaligned ? "--un-gz ${prefix}.unmapped.fastq.gz" : ''
"""
INDEX=`find -L ./ -name "*.rev.1.bt2" | sed 's/.rev.1.bt2//'`
bowtie2 \\
-x \$INDEX \\
-U $reads \\
--threads $task.cpus \\
--threads ${split_cpus} \\
$unaligned \\
$options.args \\
2> ${prefix}.bowtie2.log \\
| samtools view -@ $task.cpus $options.args2 -bhS -o ${prefix}.bam -
| samtools view -@ ${split_cpus} $options.args2 -bhS -o ${prefix}.bam -

echo \$(bowtie2 --version 2>&1) | sed 's/^.*bowtie2-align-s version //; s/ .*\$//' > ${software}.version.txt
"""
Expand All @@ -54,11 +55,11 @@ process BOWTIE2_ALIGN {
-x \$INDEX \\
-1 ${reads[0]} \\
-2 ${reads[1]} \\
--threads $task.cpus \\
--threads ${split_cpus} \\
$unaligned \\
$options.args \\
2> ${prefix}.bowtie2.log \\
| samtools view -@ $task.cpus $options.args2 -bhS -o ${prefix}.bam -
| samtools view -@ ${split_cpus} $options.args2 -bhS -o ${prefix}.bam -

if [ -f ${prefix}.unmapped.fastq.1.gz ]; then
mv ${prefix}.unmapped.fastq.1.gz ${prefix}.unmapped_1.fastq.gz
Expand Down
5 changes: 3 additions & 2 deletions software/bwa/mem/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ process BWA_MEM {
path "*.version.txt" , emit: version

script:
def split_cpus = Math.floor(task.cpus/2)
def software = getSoftwareName(task.process)
def prefix = options.suffix ? "${meta.id}${options.suffix}" : "${meta.id}"
def read_group = meta.read_group ? "-R ${meta.read_group}" : ""
Expand All @@ -36,10 +37,10 @@ process BWA_MEM {
bwa mem \\
$options.args \\
$read_group \\
-t $task.cpus \\
-t ${split_cpus} \\
\$INDEX \\
$reads \\
| samtools view $options.args2 -@ $task.cpus -bhS -o ${prefix}.bam -
| samtools view $options.args2 -@ ${split_cpus} -bhS -o ${prefix}.bam -

echo \$(bwa 2>&1) | sed 's/^.*Version: //; s/Contact:.*\$//' > ${software}.version.txt
"""
Expand Down
5 changes: 3 additions & 2 deletions software/bwamem2/mem/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ process BWAMEM2_MEM {
path "*.version.txt" , emit: version

script:
def split_cpus = Math.floor(task.cpus/2)
def software = getSoftwareName(task.process)
def prefix = options.suffix ? "${meta.id}${options.suffix}" : "${meta.id}"
def read_group = meta.read_group ? "-R ${meta.read_group}" : ""
Expand All @@ -36,10 +37,10 @@ process BWAMEM2_MEM {
bwa-mem2 mem \\
$options.args \\
$read_group \\
-t $task.cpus \\
-t ${split_cpus} \\
\$INDEX \\
$reads \\
| samtools view $options.args2 -@ $task.cpus -bhS -o ${prefix}.bam -
| samtools view $options.args2 -@ ${split_cpus} -bhS -o ${prefix}.bam -

echo \$(bwa-mem2 version 2>&1) > ${software}.version.txt
"""
Expand Down
5 changes: 3 additions & 2 deletions software/bwameth/align/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ process BWAMETH_ALIGN {
path "*.version.txt" , emit: version

script:
def split_cpus = Math.floor(task.cpus/2)
def software = getSoftwareName(task.process)
def prefix = options.suffix ? "${meta.id}${options.suffix}" : "${meta.id}"
def read_group = meta.read_group ? "-R ${meta.read_group}" : ""
Expand All @@ -36,10 +37,10 @@ process BWAMETH_ALIGN {
bwameth.py \\
$options.args \\
$read_group \\
-t $task.cpus \\
-t ${split_cpus} \\
--reference \$INDEX \\
$reads \\
| samtools view $options.args2 -@ $task.cpus -bhS -o ${prefix}.bam -
| samtools view $options.args2 -@ ${split_cpus} -bhS -o ${prefix}.bam -

echo \$(bwameth.py --version 2>&1) | cut -f2 -d" " > ${software}.version.txt
"""
Expand Down
2 changes: 0 additions & 2 deletions tests/software/bwa/mem/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
- bwa/mem
files:
- path: ./output/bwa/test.bam
md5sum: 9165508bf914baee0e6347711aa7b23a
- path: ./output/index/bwa/genome.bwt
md5sum: 0469c30a1e239dd08f68afe66fde99da
- path: ./output/index/bwa/genome.amb
Expand All @@ -24,7 +23,6 @@
- bwa/mem
files:
- path: ./output/bwa/test.bam
md5sum: 670a53bddee62d6bd14ed7adaf103e7c
- path: ./output/index/bwa/genome.bwt
md5sum: 0469c30a1e239dd08f68afe66fde99da
- path: ./output/index/bwa/genome.amb
Expand Down
2 changes: 0 additions & 2 deletions tests/software/bwamem2/mem/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
- bwamem2/mem
files:
- path: ./output/bwamem2/test.bam
md5sum: 2133c011119ea11f06f0a9b1621ba05b
- path: ./output/index/bwamem2/genome.fasta.amb
md5sum: 3a68b8b2287e07dd3f5f95f4344ba76e
- path: ./output/index/bwamem2/genome.fasta.pac
Expand All @@ -24,7 +23,6 @@
- bwamem2/mem
files:
- path: ./output/bwamem2/test.bam
md5sum: d8fadab5cef04faff1851a8162fc30b5
- path: ./output/index/bwamem2/genome.fasta.amb
md5sum: 3a68b8b2287e07dd3f5f95f4344ba76e
- path: ./output/index/bwamem2/genome.fasta.pac
Expand Down