-
Notifications
You must be signed in to change notification settings - Fork 498
removed exists true for several files in nexflow_schema #1858
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
removed exists true for several files in nexflow_schema #1858
Conversation
|
Warning Newer version of the nf-core template is available. Your pipeline is using an old version of the nf-core template: 3.0.2. For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation. |
|
good catch @kjellinjonas |
|
@nvnieuwk is there a way to have a file be optional, but if it is selected check that it exists before doing anything? |
|
Yes that's the expected behaviour. The plugin will ignore all validation if an optional file has not been given. However when the file has been given, it will do the full validation |
|
ok so removing |
|
This should be solved on a pipeline level IMO, the plugin is doing what it needs to do here. So either @kjellinjonas adds this file to the local igenomes base path (which I assume here since the germline resources path for example should exist on the s3 bucket) Or the pipeline gets updated so that these parameters only get populated when they are needed. (The first option sound the most logical to me, but I'll leave that up to you) |
removing |
|
Sure, I could add the files. However, I think it's annoying for more people than me if the pipeline requires files to be present that will not be used. How about keeping these changes, but add additional input checks in the something like: |
|
wait, I guess this would only check if the parameter is set and not if the file exists.. |
|
Option 2 would be better here then, those params shouldn't be automatically filled in if they are not being used. |
|
Not sure how to achieve that though.. Can parameters be set conditionally in |
|
These parameters are set in |
I like where this is going. But I think it feel like overcharging the |
|
Sounds good to me. However, there are currently several similar checks in So if we create a new subworkflow, I think we can move those checks as well to make it consistent. |
Make it consistent, that's my life motto |
|
Should I start the work on a new subworkflow here, or should a separate issue/PR be made for that? @maxulysse |
I'd say let's do it here, that would simplify your work, right? |
|
In utils_nfcore_sarek_pipeline, |
…nto issue1857_file_validation_fix
|
What do you think about this approach? Still a lot to be done (clean up/add test etc.) but it would be great with some input/feedback first |
| missing.collect { " - ${it[0]}" }.join('\n') | ||
| error(error_string) | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nvnieuwk is there a way to do this with the schema at the moment? If params.tools == vep, then vep_cache is required?
Would be nicer to get the failures super early and not having to wait for the head job submission
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is possible in the JSON schema spec (see if-then-else), but this is not supported by Nextflow and nf-core tools (except for nf-schema). This will also not be part of the new schema specification for the parameters we are drafting. So it's not recommended to use this feature for now. We might add it some time in the future if we get all tooling to be compatible but that's low priority right now
|
@kjellinjonas maybe for this PR we can fix the issue at hand and split out the refactoring into a different PR since it seems to be a bigger undertaking. |
|
@FriederikeHanssen Sounds good to me. What do I need to do? |
|
Remove the changes not related to the schema & changelog |
Fixes #2079 The `exists: true` validation on snpeff_cache and vep_cache parameters causes workflow launches to fail when the default S3 annotation cache paths are not accessible, even when users don't intend to use snpEff or VEP for annotation. This is the same fix applied in #1858 for other optional reference files (germline_resource, mappability, ngscheckmate_bed, sentieon_dnascope_model) but snpeff_cache and vep_cache were missed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Fixes #2079 The `exists: true` validation on snpeff_cache and vep_cache parameters causes workflow launches to fail when the default S3 annotation cache paths are not accessible, even when users don't intend to use snpEff or VEP for annotation. This is the same fix applied in #1858 for other optional reference files (germline_resource, mappability, ngscheckmate_bed, sentieon_dnascope_model) but snpeff_cache and vep_cache were missed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Co-authored-by: Claude Opus 4.5 <[email protected]> Fixes #2079 fix applied in #1858 for other optional reference files
Fix for #1857
PR checklist
nf-core pipelines lint).nextflow run . -profile test,docker --outdir <OUTDIR>).nextflow run . -profile debug,test,docker --outdir <OUTDIR>).docs/usage.mdis updated.docs/output.mdis updated.CHANGELOG.mdis updated.README.mdis updated (including new tool citations and authors/contributors).