Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@kjellinjonas
Copy link
Contributor

Fix for #1857

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/sarek branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@nf-core-bot
Copy link
Member

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.0.2.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

@maxulysse
Copy link
Member

good catch @kjellinjonas
could you update the CHANGELOG please?

@FriederikeHanssen
Copy link
Contributor

@nvnieuwk is there a way to have a file be optional, but if it is selected check that it exists before doing anything?

@nvnieuwk
Copy link
Contributor

nvnieuwk commented Apr 8, 2025

Yes that's the expected behaviour. The plugin will ignore all validation if an optional file has not been given. However when the file has been given, it will do the full validation

@FriederikeHanssen
Copy link
Contributor

ok so removing exists will do exactly that?

@nvnieuwk
Copy link
Contributor

nvnieuwk commented Apr 8, 2025

This should be solved on a pipeline level IMO, the plugin is doing what it needs to do here.

So either @kjellinjonas adds this file to the local igenomes base path (which I assume here since the germline resources path for example should exist on the s3 bucket)

Or the pipeline gets updated so that these parameters only get populated when they are needed.

(The first option sound the most logical to me, but I'll leave that up to you)

@nvnieuwk
Copy link
Contributor

nvnieuwk commented Apr 8, 2025

ok so removing exists will do exactly that?

removing exists will just ignore the existence check, do you want that in this case?

@kjellinjonas
Copy link
Contributor Author

Sure, I could add the files. However, I think it's annoying for more people than me if the pipeline requires files to be present that will not be used.

How about keeping these changes, but add additional input checks in the samplesheet_to_channel/main.nf

something like:

if (tools && tools.split(',').contains('ngschekmate')) {
        if (!ngscheckmate_bed) {
            error("No bed file was provided for running NGSCheckMate.")
        }

@FriederikeHanssen @maxulysse

@kjellinjonas
Copy link
Contributor Author

wait, I guess this would only check if the parameter is set and not if the file exists..

@nvnieuwk
Copy link
Contributor

nvnieuwk commented Apr 9, 2025

Option 2 would be better here then, those params shouldn't be automatically filled in if they are not being used.

@kjellinjonas
Copy link
Contributor Author

Not sure how to achieve that though.. Can parameters be set conditionally in igenomes.config ?

@nvnieuwk
Copy link
Contributor

nvnieuwk commented Apr 9, 2025

These parameters are set in main.nf using getGenomeAttributes, but please discuss this with the Sarek devs first which method they prefer :)

@maxulysse
Copy link
Member

Sure, I could add the files. However, I think it's annoying for more people than me if the pipeline requires files to be present that will not be used.

How about keeping these changes, but add additional input checks in the samplesheet_to_channel/main.nf

something like:

if (tools && tools.split(',').contains('ngschekmate')) {
        if (!ngscheckmate_bed) {
            error("No bed file was provided for running NGSCheckMate.")
        }

I like where this is going.

But I think it feel like overcharging the samplesheet_to_channel/main.nf.
How about we create a separate subworkflow to check out all these tools and files combinations?

@kjellinjonas
Copy link
Contributor Author

Sounds good to me. However, there are currently several similar checks in samplesheet_to_channel/main.nf (e.g. checking for missing files when mutect2 is used).

So if we create a new subworkflow, I think we can move those checks as well to make it consistent.

@maxulysse
Copy link
Member

Sounds good to me. However, there are currently several similar checks in samplesheet_to_channel/main.nf (e.g. checking for missing files when mutect2 is used).

So if we create a new subworkflow, I think we can move those checks as well to make it consistent.

Make it consistent, that's my life motto

@kjellinjonas
Copy link
Contributor Author

Should I start the work on a new subworkflow here, or should a separate issue/PR be made for that? @maxulysse

@maxulysse
Copy link
Member

Should I start the work on a new subworkflow here, or should a separate issue/PR be made for that? @maxulysse

I'd say let's do it here, that would simplify your work, right?

@kjellinjonas
Copy link
Contributor Author

In utils_nfcore_sarek_pipeline, checkPathParamList is defined and some parameters are only added if the relevant tool is used. To me this looks like a good way to define which parameters to evaluate, but I can't see that this list is actually used anywhere. Am I missing something?

@kjellinjonas
Copy link
Contributor Author

What do you think about this approach? Still a lot to be done (clean up/add test etc.) but it would be great with some input/feedback first

missing.collect { " - ${it[0]}" }.join('\n')
error(error_string)
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nvnieuwk is there a way to do this with the schema at the moment? If params.tools == vep, then vep_cache is required?
Would be nicer to get the failures super early and not having to wait for the head job submission

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is possible in the JSON schema spec (see if-then-else), but this is not supported by Nextflow and nf-core tools (except for nf-schema). This will also not be part of the new schema specification for the parameters we are drafting. So it's not recommended to use this feature for now. We might add it some time in the future if we get all tooling to be compatible but that's low priority right now

@FriederikeHanssen
Copy link
Contributor

@kjellinjonas maybe for this PR we can fix the issue at hand and split out the refactoring into a different PR since it seems to be a bigger undertaking.

@kjellinjonas
Copy link
Contributor Author

@FriederikeHanssen Sounds good to me. What do I need to do?

@FriederikeHanssen
Copy link
Contributor

Remove the changes not related to the schema & changelog

@FriederikeHanssen FriederikeHanssen merged commit b7f9a08 into nf-core:dev Apr 24, 2025
23 of 25 checks passed
pinin4fjords added a commit that referenced this pull request Dec 17, 2025
Fixes #2079

The `exists: true` validation on snpeff_cache and vep_cache parameters
causes workflow launches to fail when the default S3 annotation cache
paths are not accessible, even when users don't intend to use snpEff or
VEP for annotation.

This is the same fix applied in #1858 for other optional reference files
(germline_resource, mappability, ngscheckmate_bed, sentieon_dnascope_model)
but snpeff_cache and vep_cache were missed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
pinin4fjords added a commit that referenced this pull request Dec 17, 2025
Fixes #2079

The `exists: true` validation on snpeff_cache and vep_cache parameters
causes workflow launches to fail when the default S3 annotation cache
paths are not accessible, even when users don't intend to use snpEff or
VEP for annotation.

This is the same fix applied in #1858 for other optional reference files
(germline_resource, mappability, ngscheckmate_bed, sentieon_dnascope_model)
but snpeff_cache and vep_cache were missed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
maxulysse pushed a commit that referenced this pull request Dec 17, 2025
Co-authored-by: Claude Opus 4.5 <[email protected]>
Fixes #2079
fix applied in #1858 for other optional reference files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants