Fastq screen #3574

JPejovicApis · 2023-06-29T12:52:22Z

PR checklist

Closes #3156
Closes #3157

modules/nf-core/fastqscreen/buildfromindex/main.nf

SPPearce

This is looking really close, if the suggested change fixes the error.

modules/nf-core/fastqscreen/buildfromindex/main.nf

modules/nf-core/fastqscreen/fastqscreen/main.nf

tests/modules/nf-core/fastqscreen/buildfromindex/test.yml

tests/modules/nf-core/fastqscreen/buildfromindex/main.nf

modules/nf-core/fastqscreen/fastqscreen/meta.yml

modules/nf-core/fastqscreen/buildfromindex/main.nf

modules/nf-core/fastqscreen/buildfromindex/meta.yml

Co-authored-by: Adrien Coulier <[email protected]>

Co-authored-by: Simon Pearce <[email protected]>

SPPearce

Looking close! Just the conda check is complaining about a md5sum being different, can you swap that to a contains text?

modules/nf-core/fastqscreen/buildfromindex/meta.yml

modules/nf-core/fastqscreen/fastqscreen/main.nf

tests/modules/nf-core/fastqscreen/fastqscreen/test.yml

SPPearce

I'd like to get another review for this before merging, as it is a very tricky tool to get working.
Well done to all those involved.

tests/config/pytest_modules.yml

* new module picard/scatterintervalsbyns * update annotsv to 3.3.6 * update installannotations * new module gem2/gem2bedmappability * add sizes output * update meta * review comments

* new module picard/scatterintervalsbyns * update annotsv to 3.3.6 * update installannotations * update genmap modules * revert label changes index

* update modules * update uri * review suggestions * update germlinecnvcaller

* update modules * update uri * review suggestions * update germlinecnvcaller * update params

* Add singularity.registry = 'quay.io' for tests * Bump min NF version to 23.04.0 * Move registry definitions to top-level config scope --------- Co-authored-by: Gregor Sturm <[email protected]>

* add README * Implement cellranger autorename * Fix container paths * Various fixes * Revert "Fix container paths" This reverts commit 4163e2b. * Update modules/nf-core/cellranger/count/templates/cellranger_count.py Co-authored-by: Kevin L. Keys <[email protected]> * Apply suggestions from code review Co-authored-by: Kevin L. Keys <[email protected]> * Include container registry * Update modules/nf-core/cellranger/count/main.nf Co-authored-by: Maxime U Garcia <[email protected]> * Update documentation --------- Co-authored-by: Kevin L. Keys <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]>

* Bump taxpasta version * Update modules/nf-core/taxpasta/merge/main.nf Co-authored-by: Moritz E. Beber <[email protected]> * Update modules/nf-core/taxpasta/standardise/main.nf Co-authored-by: Moritz E. Beber <[email protected]> * Update modules/nf-core/taxpasta/merge/main.nf Co-authored-by: Moritz E. Beber <[email protected]> --------- Co-authored-by: Moritz E. Beber <[email protected]>

…into fastq_screen

SPPearce · 2024-04-29T22:40:07Z

I've ported the tests over to nf-test.
I haven't manage to sort out the syntax to check the fastqscreen/fastqscreen output channel though, that is eluding me. It has three files:

test_fq_screen/test_1_screen.txt
test_fq_screen/test_1_screen.png
test_fq_screen/test_1_screen.html

Of which the .txt and .png files are stable, the .html isn't. But I can't manage to get it to snapshot properly without some kind of error.

adamrtalbot

For the variable *.txt, *.png and *.html files, capture them as separate outputs then it will be easier to separate them in the process.

e.g. in the output block:

tuple val(meta), path("*.txt"), emit: txt
tuple val(meta), path("*.png"), emit: png
tuple val(meta), path("*.html"), emit: html

in the nf-test:

assert snapshot(process.out.txt, process.out.png).match()

modules/nf-core/fastqscreen/buildfromindex/tests/main.nf.test

SPPearce · 2024-04-30T14:06:06Z

Grrr, the other two files have consistent md5sums on conda and singularity, but they are different to docker...

adamrtalbot · 2024-05-01T10:40:52Z

Grrr, the other two files have consistent md5sums on conda and singularity, but they are different to docker...

That's very unusual, I would expect Singularity and Docker to be the same.

Might have to accept whatever testing is possible here (process success and file exists), plus any easy file contents checking that makes sense.

SPPearce · 2024-05-02T07:25:01Z

Grrr, the other two files have consistent md5sums on conda and singularity, but they are different to docker...

That's very unusual, I would expect Singularity and Docker to be the same.

Might have to accept whatever testing is possible here (process success and file exists), plus any easy file contents checking that makes sense.

Yes, I just swapped to file existance.

adamrtalbot

It all looks good, but there's something up with the indentation and perhaps a few styling choices, but the code itself is fine.

adamrtalbot · 2024-05-02T08:22:37Z

modules/nf-core/fastqscreen/buildfromindex/tests/main.nf.test

+            process {
+                """
+                input[0] =  Channel.from([
+                                    [[id: "sarscov2"], file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)],
+                                    [[id: "human"]   , file(params.test_data['homo_sapiens']['genome']['genome_21_fasta'], checkIfExists: true)]
+                            ])
+                """


The indentation seems a bit weird here?

To be honest, I don't know how we are supposed to ident things like this

modules/nf-core/fastqscreen/buildfromindex/tests/main.nf.test

adamrtalbot · 2024-05-02T08:23:55Z

modules/nf-core/fastqscreen/fastqscreen/tests/main.nf.test

+                input[0] =  [[ id:'test', single_end:true ],
+                             [file(params.test_data['sarscov2']['illumina']['test_1_fastq_gz'], checkIfExists: true) ]
+                            ]


Definitely something funky with the indentation.

modules/nf-core/fastqscreen/fastqscreen/tests/main.nf.test

Co-authored-by: Adam Talbot <[email protected]>

snesic added 2 commits June 29, 2023 14:29

use id as index name

92c8a67

prettier yml

13fd77b

JPejovicApis requested a review from ewels June 29, 2023 12:52

nevinwu mentioned this pull request Aug 25, 2023

new module: FastQ-Screen #3157

Closed

Aratz reviewed Sep 6, 2023

View reviewed changes

modules/nf-core/fastqscreen/buildfromindex/main.nf Show resolved Hide resolved

SPPearce reviewed Sep 29, 2023

View reviewed changes

SPPearce and others added 12 commits September 29, 2023 06:20

Update modules/nf-core/fastqscreen/buildfromindex/main.nf

84551ed

Co-authored-by: Adrien Coulier <[email protected]>

Merge branch 'master' into fastq_screen

e1e5346

remove comment

99c7123

Update tests/modules/nf-core/fastqscreen/buildfromindex/main.nf

ac168d7

Co-authored-by: Simon Pearce <[email protected]>

Update modules/nf-core/fastqscreen/fastqscreen/meta.yml

b105024

Co-authored-by: Simon Pearce <[email protected]>

Update tests/modules/nf-core/fastqscreen/buildfromindex/main.nf

be1390b

Co-authored-by: Simon Pearce <[email protected]>

change genome to database

9088b35

rename input

9aab90d

remove meta2

3a1e332

update md5sum

c47babb

correct container

d667928

Update main.nf

b4c09ad

SPPearce reviewed Oct 3, 2023

View reviewed changes

modules/nf-core/fastqscreen/buildfromindex/meta.yml Outdated Show resolved Hide resolved

modules/nf-core/fastqscreen/fastqscreen/main.nf Outdated Show resolved Hide resolved

tests/modules/nf-core/fastqscreen/fastqscreen/test.yml Outdated Show resolved Hide resolved

snesic added 2 commits October 4, 2023 09:15

Update meta.yml

f2ffc4e

remove inplace in sed to work on macos and linux

cf4f2d6

SPPearce approved these changes Oct 4, 2023

View reviewed changes

maxulysse reviewed Oct 6, 2023

View reviewed changes

tests/config/pytest_modules.yml Outdated Show resolved Hide resolved

nvnieuwk and others added 7 commits October 10, 2023 14:56

Gem2/gem2bedmappability (nf-core#3572)

3621748

* new module picard/scatterintervalsbyns * update annotsv to 3.3.6 * update installannotations * new module gem2/gem2bedmappability * add sizes output * update meta * review comments

Update genmap modules (nf-core#3579)

1044859

* new module picard/scatterintervalsbyns * update annotsv to 3.3.6 * update installannotations * update genmap modules * revert label changes index

Gcnvcaller modules (nf-core#3581)

825e7d6

* update modules * update uri * review suggestions * update germlinecnvcaller

Update germlinecnvcaller (nf-core#3584)

cee34e4

* update modules * update uri * review suggestions * update germlinecnvcaller * update params

Add singularity.registry = 'quay.io' for tests (nf-core#3499)

96f9900

* Add singularity.registry = 'quay.io' for tests * Bump min NF version to 23.04.0 * Move registry definitions to top-level config scope --------- Co-authored-by: Gregor Sturm <[email protected]>

snesic force-pushed the fastq_screen branch from cf4f2d6 to b0a5e27 Compare October 10, 2023 12:59

snesic closed this Oct 10, 2023

snesic force-pushed the fastq_screen branch from b0a5e27 to efdda7b Compare October 10, 2023 13:36

Merge branch 'fastq_screen' of https://github.com/JPejovicApis/modules …

1a7266d

…into fastq_screen

SPPearce reopened this Oct 10, 2023

snesic added 3 commits October 10, 2023 17:56

Merge branch 'fastq_screen' of https://github.com/JPejovicApis/modules …

47fc6fa

…into fastq_screen

correct pytest_modules and prettier

0a68d65

Merge branch 'nf-core:master' into fastq_screen

f6d91b9

edmundmiller requested review from a team and edmundmiller October 19, 2023 16:26

edmundmiller removed their request for review February 6, 2024 15:35

SPPearce added 3 commits April 29, 2024 21:59

Merge branch 'master' into fastq_screen

948be88

Add nf-test for buildfromindex

8f45c25

Add nf-test for fastqscreen

5c8e9de

adamrtalbot reviewed Apr 30, 2024

View reviewed changes

modules/nf-core/fastqscreen/buildfromindex/tests/main.nf.test Outdated Show resolved Hide resolved

Update nf-test to check stable files

dc2cf3f

SPPearce added 2 commits April 30, 2024 14:13

Move setup block to shared

11e813e

Update nftest

fb79246

Update test

91f2996

adamrtalbot approved these changes May 2, 2024

View reviewed changes

SPPearce and others added 2 commits May 2, 2024 09:52

Update modules/nf-core/fastqscreen/buildfromindex/tests/main.nf.test

a62acc0

Co-authored-by: Adam Talbot <[email protected]>

Update modules/nf-core/fastqscreen/fastqscreen/tests/main.nf.test

457921b

Co-authored-by: Adam Talbot <[email protected]>

SPPearce added this pull request to the merge queue May 3, 2024

Merged via the queue into nf-core:master with commit e1316cd May 3, 2024

SPPearce mentioned this pull request Jul 31, 2024

Updated test data paths batch 8 #6065

Merged

17 tasks

Fastq screen #3574

Fastq screen #3574

Uh oh!

Conversation

JPejovicApis commented Jun 29, 2023 • edited by FelixKrueger Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR checklist

Uh oh!

Uh oh!

SPPearce left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SPPearce left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SPPearce left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SPPearce commented Apr 29, 2024

Uh oh!

adamrtalbot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SPPearce commented Apr 30, 2024

Uh oh!

adamrtalbot commented May 1, 2024

Uh oh!

SPPearce commented May 2, 2024

Uh oh!

adamrtalbot left a comment

Choose a reason for hiding this comment

Uh oh!

adamrtalbot May 2, 2024

Choose a reason for hiding this comment

Uh oh!

SPPearce May 2, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

adamrtalbot May 2, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

JPejovicApis commented Jun 29, 2023 •

edited by FelixKrueger

Loading