Codestin Search App

jorgee · 2026-01-28T12:25:34Z

This pull request introduces a new module command-line interface for managing Nextflow modules, including installing, listing, and publishing modules. It adds a new module command with several subcommands, updates the CLI infrastructure to support unknown options for this command, and brings in new dependencies required for module packaging and extraction.

Major new features and CLI enhancements:

Added a new CmdModule class implementing the nextflow module command, with subcommands for install, list, publish, run, remove, and search. This includes full CLI argument parsing, usage/help handling, and subcommand delegation.
ModuleRun extends CmdRun installing the module and calling CmdRun with the downloaded module.

Launcher and CLI infrastructure updates:

Registered the new CmdModule command in the main Launcher class so it is available as a top-level command.
Modified the CLI parser in Launcher to allow unknown options for the module command and to pass these unknown options to the command implementation. [1] [2]
Updated the base CmdBase class to support storing and retrieving unknown CLI options, enabling more flexible argument handling for subcommands.

Dependency management:

Added org.apache.commons:commons-compress as a dependency for handling tar.gz extraction, which is necessary for module packaging and publishing.

Building and Testing
Current CI is failing becuase it requires a local development registry instance with modules API implemented. You will need it to update build.gradle to updated npr-api package with the new module API classes. Either with includeBuild or a Maven local repository. You will also need to deploy the registry locally and configure the registry field in nextflow.config to test the new CLI commans

Open Issue and TODOs:

Module version temporary in nextflow_spec.json for simplicity.
Print module run outputs
Remote module inclusion
Add documentation
Extend unit and integration tests

jorgee · 2026-01-30T20:04:22Z

Some updates:

1.- I have added the print of process outputs when finishing the process execution.

2.- I have investigated the remote module inclusion and a possible implemetation:
When using the syntax parser V1, it is easy to implement the module resolution at IncludeDef. We can check if the module is installed and install if not exists.

However, when using syntax parser V2, the problem is that visitors and resolvers used for parsing are at nf-lang. So, even if I decouple the classes to download modules and import at nf-lang, there is no way to access the session, config builder, and we will add the npr-api classes. Not sure if it would be a good solution

So, what I have done is the following:

In the include resolver and visitor, I just convert the @scope/module to the expected path and check if it exists. If there is no local copy of the module, it fails at compile time and suggest the user to install it. If there is a local installation, it checks the versions and integrity at IncludeDef.
I could also check the installation of the declared modules at CmdRun, then it will only fail if a module is used and not declared.

bentsherman · 2026-02-02T15:43:40Z

@jorgee let's move the remote module inclusion piece out of this PR. you can put it in a separate PR if you want, either way Paolo will experiment with it separately

And for the module version config, in the interest of keeping the POC simple so that we can iterate quicker, let's keep the module versions in the JSON file. Paolo can also experiment with this separately if he wants

jorgee · 2026-02-06T20:04:54Z

Updated the branch as @bentsherman suggested

Modules version temporary in nextflow_spec.json
Remote inclusion PoC in Remote module inclusion #6815
Added docs and tests
It will be ready once changes in registry are aplied and new npr-api version generated

pditommaso · 2026-02-09T17:40:33Z

I was giving a try to this, but cannot compile it

> Task :nextflow:compileGroovy
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
startup failed:
/Users/pditommaso/Projects/nextflow/modules/nextflow/src/main/groovy/nextflow/module/ModuleRegistryClient.groovy: 24: unable to resolve class io.seqera.npr.api.schema.v1.ModuleRelease
 @ line 24, column 1.
   import io.seqera.npr.api.schema.v1.ModuleRelease
   ^

/Users/pditommaso/Projects/nextflow/modules/nextflow/src/main/groovy/nextflow/module/ModuleRegistryClient.groovy: 25: unable to resolve class io.seqera.npr.api.schema.v1.PublishModuleResponse
 @ line 25, column 1.
   import io.seqera.npr.api.schema.v1.PublishModuleResponse
   ^

/Users/pditommaso/Projects/nextflow/modules/nextflow/src/main/groovy/nextflow/module/ModuleRegistryClient.groovy: 205: unable to resolve class io.seqera.npr.api.schema.v1.ModuleRelease
 @ line 205, column 37.
                return new GsonEncoder<ModuleRelease>() {}.decode(body)
                                       ^

/Users/pditommaso/Projects/nextflow/modules/nextflow/src/main/groovy/nextflow/module/ModuleRegistryClient.groovy: 472: unable to resolve class io.seqera.npr.api.schema.v1.PublishModuleResponse
 @ line 472, column 36.
               return new GsonEncoder<PublishModuleResponse>() {}.decode(body)
                                      ^

/Users/pditommaso/Projects/nextflow/modules/nextflow/src/main/groovy/nextflow/module/ModuleRegistryClient.groovy: 205: unable to resolve class io.seqera.npr.api.schema.v1.ModuleRelease
 @ line 205, column 37.
                return new GsonEncoder<ModuleRelease>() {}.decode(body)
                                       ^

/Users/pditommaso/Projects/nextflow/modules/nextflow/src/main/groovy/nextflow/module/ModuleRegistryClient.groovy: 205: unable to resolve class io.seqera.npr.api.schema.v1.ModuleRelease
 @ line 205, column 37.
                return new GsonEncoder<ModuleRelease>() {}.decode(body)
                                       ^

/Users/pditommaso/Projects/nextflow/modules/nextflow/src/main/groovy/nextflow/module/ModuleRegistryClient.groovy: 472: unable to resolve class io.seqera.npr.api.schema.v1.PublishModuleResponse
 @ line 472, column 36.
               return new GsonEncoder<PublishModuleResponse>() {}.decode(body)
                                      ^

/Users/pditommaso/Projects/nextflow/modules/nextflow/src/main/groovy/nextflow/module/ModuleRegistryClient.groovy: 472: unable to resolve class io.seqera.npr.api.schema.v1.PublishModuleResponse
 @ line 472, column 36.
               return new GsonEncoder<PublishModuleResponse>() {}.decode(body)
                                      ^

8 errors

jorgee · 2026-02-10T09:57:13Z

To test this branch you need to do the following:

Clone the registry repo and clone branch 251215-module-binary-registry-fixes
Build and publish the npr-api version 0.20.1 in Maven local

$ make build
$ ./gradlew :npr-api:publishToMavenLocal

Deploy a local docker registry to store modules
$ docker run -d --name docker-registry -p 5010:5000 registry:latest
Run the Nextflow registry
$ bash run.sh
Clone Nextflow repo and checkout this branch
Include the Maven local in the build.gradle file

         mavenCentral()
         maven { url 'https://repo.eclipse.org/content/groups/releases' }
         maven { url 'https://oss.sonatype.org/content/repositories/snapshots' }
+        mavenLocal()
         maven { url = "https://s3-eu-west-1.amazonaws.com/maven.seqera.io/releases" }
         maven { url = "https://s3-eu-west-1.amazonaws.com/maven.seqera.io/snapshots" }

Build Nextflow as usual

Signed-off-by: jorgee <[email protected]>

…te to v1 registry and other required fixes Signed-off-by: jorgee <[email protected]>

Signed-off-by: jorgee <[email protected]>

jorgee · 2026-02-19T11:10:35Z

Update:

Add module info command
Update to registry API to v1
/api path expected in registry config URL
search only prints name and description (full for -json)

To test, you must run with registry branch fix-module-release-metadata

…ntation

Signed-off-by: Paolo Di Tommaso <[email protected]>

Signed-off-by: jorgee <[email protected]>

… publish (-registry -> config.url -> default) Signed-off-by: jorgee <[email protected]>

jorgee · 2026-02-27T12:28:51Z

Updated docs and other fixes. To test in dev, add the following lines in nextflow.config

registry {
     url='https://registry-dev.nextflow.io/api'
     apiKey='YOUR_REGISTRY_TOKEN'
}

Signed-off-by: jorgee <[email protected]>

…ntation

Signed-off-by: jorgee <[email protected]>

pditommaso · 2026-03-05T08:37:09Z

Module System Code Review

CRITICAL (Must Fix)

1. Logic Bug — `CmdModuleSearch` validation (dead code)

CmdModuleSearch.groovy:64 — if (!args && args.size() != 1) is impossible: if !args is true, && short-circuits. Should be ||.

2. Inverted Condition — `CmdModuleInfo` (wrong log + potential NPE)

CmdModuleInfo.groovy:102-109 — Logs "No metadata found" when metadata IS present. Additionally, if fetchRelease throws, release stays null and the code falls through to printFormattedInfo which will NPE.

3. `e.printStackTrace()` in production code

ModuleRegistryClient.groovy:138 — Writes directly to stderr, bypassing the logging framework. Replace with log.debug or log.error.

IMPORTANT (Should Fix)

5. Checksum header name mismatch between code and tests

Production code checks X-NF-Module-Checksum, but tests stub X-Checksum. Tests pass but don't actually validate the real header.

6. `Files.list()` / `Files.walk()` stream resource leaks

ModuleStorage.groovy:126,171,409, ModuleChecksum.groovy:54, CmdModulePublish.groovy:222 — These streams must be closed (try-with-resources or .withCloseable{}). Compare with ModuleStorage.groovy:345 where this is done correctly.

7. `removeModuleEntry` always returns true

PipelineSpec.groovy:82-96 — Doesn't check if the key actually existed before returning true. Map.remove() returns the previous value (or null); the return value should be checked.

8. Bundle sent as raw bytes in JSON

CmdModulePublish.groovy:129-135 — byte[] in Gson JSON will be huge/inefficient. Consider multipart upload.

9. `Files.readAllBytes()` for checksum computation

ModuleChecksum.groovy:67,131 — Loads entire files into memory. Use DigestInputStream or read in chunks for large files.

10. Missing `ModulesConfig` in `extensions.idx`

RegistryConfig is registered but ModulesConfig is not — may not be discovered by the config scope mechanism.

SUGGESTIONS (Nice to Have)

Documentation code example not working #11 Typo: "diretory" → "directory" in CmdModulePublish.groovy:253
Error resuming process with output value and command defined on multiple lines #12 Unicode checkmark in CLI output may not render on all terminals
Show a more detailed error message when hitting GitHub rate limits #13 CmdModuleRun extends CmdRun exposes all CmdRun params — consider composition
Very minor typo fixes #14 Duplicate config-building boilerplate across all CLI commands — extract to shared method

Strengths

Zip slip / tar slip protection is correctly implemented
Good checksum verification with cleanup on mismatch
Solid test coverage with WireMock integration tests
Clean CLI subcommand pattern consistent with existing CmdPlugin
Proper Apache 2.0 headers, @CompileStatic, @Slf4j, Groovy idioms throughout

pditommaso · 2026-03-05T08:37:45Z

Apologies the spam, but it looks valid points ☝️

Signed-off-by: jorgee <[email protected]>

jorgee · 2026-03-05T13:39:59Z

Update:

Addressed critical and important fixes except 8 (not required because bundle is limited to 1MB not compressed) and 10 (depends on where we decide to write module versions)
Change -json by -output
Add module URL in info command

TODO/to be decided:

remove flags
remove '@' (I will create another PR as it require changes also in remote inclusion)
module versions in nextflow_spec.json or nextflow.config

Signed-off-by: jorgee <[email protected]>

Signed-off-by: Ben Sherman <[email protected]>

Signed-off-by: jorgee <[email protected]>

jorgee · 2026-03-06T19:18:48Z

I have removed -keep-config for module remove. I have also changed its behaviour:

It fails when folder exists and .module-info doesn't exist or when there are modifications
Added the -force flag to remove in above cases
When removing with -keep-files, it removes the entry in spec and the .module-info file

christopher-hakkaart

Minor suggestions

Co-authored-by: Chris Hakkaart <[email protected]> Signed-off-by: Jorge Ejarque <[email protected]>

Signed-off-by: jorgee <[email protected]>

bentsherman · 2026-03-10T13:22:39Z

Merging into the main PR since we've resolved all of the requirements, let's continue to test and refine there

@constraint

* Add Nextflow Development Constitution Introduce comprehensive development constitution documenting core principles and practices for Nextflow development including modular architecture, test-driven quality assurance, dataflow programming model, licensing compliance, DCO requirements, semantic versioning, and Groovy code standards. The constitution codifies existing best practices from CLAUDE.md and CONTRIBUTING.md to provide clear governance and quality standards for the project. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Module adr v1 Signed-off-by: Paolo Di Tommaso <[email protected]> * Module adr v2 Signed-off-by: Paolo Di Tommaso <[email protected]> * ADR: Nextflow remote modules system v2.1 Signed-off-by: Paolo Di Tommaso <[email protected]> * Add detailed CLI command descriptions to module system ADR [ci skip] - Add comprehensive documentation for all module CLI commands - Add `nextflow module run` command for standalone module execution - Remove `module update` command to simplify the design - Use single-dash prefix for Nextflow options, double-dash for module inputs - Remove @ prefix from scope in CLI commands (keep only in DSL syntax) Signed-off-by: Paolo Di Tommaso <[email protected]> * Simplify module spec: unify dependencies under requires - Remove separate `dependencies` and `components` fields - Expand `requires` to include: - `nextflow`: version constraint (unchanged) - `plugins`: array with name@constraint syntax - `modules`: array of module dependencies - `workflows`: array of workflow dependencies - Unified version constraint syntax: `[scope/]name[@constraint]` - Mark `components` as deprecated (use requires.modules) - Update all examples in ADR and schema Signed-off-by: Paolo Di Tommaso <[email protected]> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Add structured tool arguments to module spec schema - Add `args` property to tools section for type-safe argument configuration - Define toolArgSpec with flag, type, description, default, enum, required - Support implicit variable `tools.<toolname>.args.<argname>` returning formatted flag+value (e.g., "-K 100000000") - Support `tools.<toolname>.args` to return all args concatenated - Document deprecation of ext.args/ext.args2/ext.args3 pattern - Update ADR with Tool Arguments Configuration section and appendix [ci skip] Signed-off-by: Paolo Di Tommaso <[email protected]> * Update adr/20251114-module-system.md [ci skip] Co-authored-by: Jorge Ejarque <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Update adr/20251114-module-system.md [ci skip] Co-authored-by: Jorge Ejarque <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Update adr/20251114-module-system.md [ci skip] Co-authored-by: Jorge Ejarque <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Update adr/20251114-module-system.md [ci skip] Expand deprecation notice to cover all ext.* custom directives Signed-off-by: Paolo Di Tommaso <[email protected]> * Fix inconsistencies in module system ADR [ci skip] - Use consistent module path format with version: modules/@scope/name@version/ - Fix directory structure example: samtools-view -> samtools/view - Standardize on 'license' spelling (American English) - Fix author -> authors (plural array format) Signed-off-by: Paolo Di Tommaso <[email protected]> * Fix registry API path in comparison table [ci skip] Remove @ prefix from scope in API path to match API definition Signed-off-by: Paolo Di Tommaso <[email protected]> * Fix checksum format: sha256- -> sha256: [ci skip] Signed-off-by: Paolo Di Tommaso <[email protected]> * Update module API to match registry implementation [ci skip] - Update API endpoints to match seqeralabs/nextflow-registry#266 - Use /api/modules base path (no v1 prefix) - Use single {name} parameter with namespace (e.g., "nf-core/fastqc") - Add separate /releases endpoint - Simplify publish to single POST endpoint Signed-off-by: Paolo Di Tommaso <[email protected]> * Update adr [ci skip] Signed-off-by: Paolo Di Tommaso <[email protected]> * Simplify module storage and verification model [ci skip] - Remove checksums from nextflow.config (registry is source of truth) - Use single version per module locally (no version in directory path) - Add .checksum file in module directory for integrity verification - Simplify freeze command to only pin transitive dependency versions - Checksum mismatch reports warning instead of automatic re-download - Remove resolved open question about checksum verification Signed-off-by: Paolo Di Tommaso <[email protected]> * Update adr/20251114-module-system.md [ci skip] Co-authored-by: Jorge Ejarque <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Clarify module resolution rules for version changes vs local modifications [ci skip] - Added Resolution Rules table clearly specifying behavior for each scenario: - Version change (local unmodified): automatically replace with declared version - Local modification (checksum mismatch): warn and protect local changes - Use -force flag to override locally modified modules - Updated install command behavior to reflect checksum verification - Updated Technical Details with expanded resolution flow Signed-off-by: Paolo Di Tommaso <[email protected]> * ADR v2.3: Fix resolution rules consistency and add version entry [ci skip] - Fixed Resolution Rules table: locally modified modules will NOT be replaced unless -force is used (was incorrectly saying "will be replaced") - Added Version 2.3 changelog entry documenting: - Resolution Rules table with clear behavior matrix - Local modification protection with -force flag - Simplified storage model (single version per module) - .checksum file for fast integrity verification Signed-off-by: Paolo Di Tommaso <[email protected]> * Fix Version 2.1 date typo: 2025 -> 2024 [ci skip] Signed-off-by: Paolo Di Tommaso <[email protected]> * Add module system client specification [ci skip] Specification for Nextflow module system client implementation based on ADR 20251114-module-system.md. Covers: P1 (Core): - Install and use registry modules via @scope/name syntax - Run modules directly from CLI without wrapper workflow - Structured tool arguments replacing ext.args pattern P2 (Important): - Module version management and freeze command - Module integrity protection with checksum validation P3 (Nice to have): - Remove module command - Search and discover modules - Publish module to registry Registry backend is out of scope (assumed implemented). Signed-off-by: Paolo Di Tommaso <[email protected]> * Remove transitive dependency resolution from module system [ci skip] - Remove `freeze` command from CLI - Remove transitive dependency install behavior - Remove orphaned transitive dependency removal from `remove` command - Update rationale and consequences sections - Simplify dependency resolution flow - Update ADR to version 2.4 Co-Authored-By: Claude Opus 4.5 <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Update adr/20251114-module-system.md [ci skip] Co-authored-by: Jorge Ejarque <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Update adr/20251114-module-system.md [ci skip] Co-authored-by: Phil Ewels <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Update modules plan [ci skip] Signed-off-by: Paolo Di Tommaso <[email protected]> * Remove tools config [ci skip] Signed-off-by: Paolo Di Tommaso <[email protected]> * Simplify module system requires definition [ci skip] - Remove plugins, modules, subworkflows from requires block - Spec now focused on process modules only - Dependencies managed via nextflow.config Signed-off-by: Paolo Di Tommaso <[email protected]> * Replace tool arguments with module parameters in ADR [ci skip] - Module parameters defined in meta.yaml params section with name, type, description, and example attributes - Removed tools args property and specification - Updated schema and examples throughout Co-Authored-By: Claude Opus 4.5 <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Update specs + adr [ci skip] Signed-off-by: Paolo Di Tommaso <[email protected]> * Resolve some conflicts with nextflow-io/schemas#10 Signed-off-by: Ben Sherman <[email protected]> * Update adr/20251114-module-system.md [ci skip] Co-authored-by: Ben Sherman <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * Extract module parameters to separate spec [ci skip] Co-Authored-By: Claude Opus 4.5 <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * docs: clarify path name for staged task inputs (#6869) Co-authored-by: Ben Sherman <[email protected]> * Add devcontainer (#6792) Co-authored-by: Phil Ewels <[email protected]> Co-authored-by: Ben Sherman <[email protected]> * Fix vertical alignment in timeline report (#6794) * Ensure main script is first in the WorkflowRun lineage record (#6845) * Add whitespace rules to .editorconfig (#5606) * docs: Update your first script to use outputs (#6500) * Update AWS CLI install docs to bypass conda install TOS prompt (#6685) * Clarify onError and onComplete handler descriptions (#6709) [ci fast] * Use explicit memory units for LSF executor (#5217) Signed-off-by: Ben Sherman <[email protected]> * Treat LSF job status UNKWN as HOLD (#5756) * Allow boolean params to implicitly default to false (#6764) * Sched core implementation alpha1 (#6242) Signed-off-by: Paolo Di Tommaso <[email protected]> Co-authored-by: Claude Opus 4.5 <[email protected]> Co-authored-by: Lorenzo Fontana <[email protected]> * Revert version 26.01.1-edge Signed-off-by: Paolo Di Tommaso <[email protected]> * Bump copyright 2026 Signed-off-by: Paolo Di Tommaso <[email protected]> * Handle Azure Batch ActiveJobAndScheduleQuotaReached with retry (#6874) * Handle Azure Batch ActiveJobAndScheduleQuotaReached with retry (#5575) Add retry logic when Azure Batch returns HTTP 409 with ActiveJobAndScheduleQuotaReached error code during job creation, instead of failing immediately. Configurable via azure.batch.maxJobQuotaRetries (default 3) and azure.batch.jobQuotaRetryDelay (default 2 min). Generated by Claude Code Signed-off-by: adamrtalbot <[email protected]> * Refactor job quota retry to use Failsafe RetryPolicy Signed-off-by: Paolo Di Tommaso <[email protected]> Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> --------- Signed-off-by: adamrtalbot <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> Co-authored-by: Paolo Di Tommaso <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> * [ci fast] Add agent output mode (NXF_AGENT_MODE) Add AI agent-friendly output mode activated via NXF_AGENT_MODE=1. When enabled, Nextflow replaces the interactive ANSI progress display with minimal, structured tagged lines optimized for AI agent context windows: [PIPELINE] name version | profile=X [WORKDIR] /path/to/work [PROCESS hash] name (tag) [WARN] deduplicated warning [ERROR] name with exit/cmd/stderr/workdir [SUCCESS|FAILED] completed=N failed=N cached=N Key changes: - LogObserver interface as common abstraction for AnsiLogObserver and AgentLogObserver - Session holds a single logObserver field instead of two concrete fields - DefaultObserverFactory creates both observer types using the same pattern - AgentLogObserver filters hash-prefixed log lines to avoid duplicating [PROCESS] output - Workflow output (view operator) passes through to stdout in agent mode - Renamed env var from NXF_AGENT to NXF_AGENT_MODE - Added NXF_AGENT_MODE to environment variables documentation * Bump [email protected] Signed-off-by: Paolo Di Tommaso <[email protected]> * Bump [email protected] Signed-off-by: Paolo Di Tommaso <[email protected]> * Bump [email protected] Signed-off-by: Paolo Di Tommaso <[email protected]> * Bump [email protected] Signed-off-by: Paolo Di Tommaso <[email protected]> * Bump [email protected] Signed-off-by: Paolo Di Tommaso <[email protected]> * Bump [email protected] nextflowVersion Signed-off-by: Paolo Di Tommaso <[email protected]> * Bump [email protected] nextflowVersion and README Signed-off-by: Paolo Di Tommaso <[email protected]> * Fix AgentLogObserver println clash with Groovy GDK method Signed-off-by: Paolo Di Tommaso <[email protected]> * Add Summary section to plugin README files Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Paolo Di Tommaso <[email protected]> * [release] [ci fast] Nextflow version 26.02.0-edge Signed-off-by: Paolo Di Tommaso <[email protected]> * Fix docs [ci skip] Signed-off-by: Paolo Di Tommaso <[email protected]> * Add logStreamId to Tower task record (#6877) Propagate the log stream identifier from TraceRecord to the Tower task record so that Seqera Platform can link tasks to their cloud log streams. Signed-off-by: Paolo Di Tommaso <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> * Don't include container in task trace when containers are not enabled (#6824) * Propagate errors from workflow outputs (#6876) * Fix type detection of CLI params in v2 config parser (#6765) * Add NVMe disk allocation and diskMountPath support (#6879) [ci fast] * Add NVMe disk allocation and diskMountPath support for Seqera executor For task/node allocation the default is /tmp; for nvme allocation the default is the server-configured NVMe mount path (typically /scratch). Signed-off-by: Paolo Di Tommaso <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> * Version 2.7: rename `.checksum` to `.module-info`, remove `@` prefix from module scopes, remove version pinning from `nextflow.config` Signed-off-by: Ben Sherman <[email protected]> * Module System Implementation (#6768) Signed-off-by: jorgee <[email protected]> * minor edits Signed-off-by: Ben Sherman <[email protected]> * Cleanup module run output Signed-off-by: Ben Sherman <[email protected]> * Fix meta.yaml -> meta.yml, module manifest -> module spec Signed-off-by: Ben Sherman <[email protected]> * Fix failing test Signed-off-by: Ben Sherman <[email protected]> * fix tests Signed-off-by: jorgee <[email protected]> * modify module list to skip subdirectories of directories that already contains the .module-info. This folders can't be a module Signed-off-by: jorgee <[email protected]> * Update docs Signed-off-by: Ben Sherman <[email protected]> * Apply suggestion from @pditommaso [ci skip] Signed-off-by: Paolo Di Tommaso <[email protected]> --------- Signed-off-by: Paolo Di Tommaso <[email protected]> Signed-off-by: Ben Sherman <[email protected]> Signed-off-by: adamrtalbot <[email protected]> Signed-off-by: jorgee <[email protected]> Co-authored-by: Claude <[email protected]> Co-authored-by: Jorge Ejarque <[email protected]> Co-authored-by: Phil Ewels <[email protected]> Co-authored-by: Ben Sherman <[email protected]> Co-authored-by: Abi <[email protected]> Co-authored-by: Peter Kneale <[email protected]> Co-authored-by: Chris Hakkaart <[email protected]> Co-authored-by: Rintze Zelle <[email protected]> Co-authored-by: Colin J. Brislawn <[email protected]> Co-authored-by: Lorenzo Fontana <[email protected]> Co-authored-by: Adam Talbot <[email protected]> Co-authored-by: Edmund Miller <[email protected]> Co-authored-by: jorgee <[email protected]>

bentsherman reviewed Jan 28, 2026

View reviewed changes

Comment thread modules/nextflow/src/main/groovy/nextflow/module/ModuleManifest.groovy Outdated

bentsherman reviewed Jan 28, 2026

View reviewed changes

Comment thread modules/nextflow/src/main/groovy/nextflow/util/NextflowSpecFile.groovy Outdated

jorgee force-pushed the 251117-module-system-implementation branch from 23fc2f2 to 7dd3e52 Compare February 5, 2026 12:45

jorgee added 4 commits February 10, 2026 11:05

Add first implementation of module CLI commands

f5b6b86

Signed-off-by: jorgee <[email protected]>

add print of process outputs

f5822a5

Signed-off-by: jorgee <[email protected]>

rename ModuleSpec and PipelineSpec

24a741f

Signed-off-by: jorgee <[email protected]>

add documentation and tests

2432e90

Signed-off-by: jorgee <[email protected]>

jorgee force-pushed the 251117-module-system-implementation branch from 1785870 to 2432e90 Compare February 10, 2026 10:05

jorgee added 2 commits February 10, 2026 11:37

add registry config as config extension

31bf49c

Signed-off-by: jorgee <[email protected]>

fix NPE in nextflow CLI help

b091d73

Signed-off-by: jorgee <[email protected]>

jorgee mentioned this pull request Feb 11, 2026

Remote module inclusion #6815

Merged

pditommaso force-pushed the 251117-module-system-implementation branch from f4095a7 to b091d73 Compare February 16, 2026 23:17

This comment was marked as outdated.

Sign in to view

jorgee added 2 commits February 19, 2026 11:59

add module info and expect /api path in the config registry url, upda…

a03f630

…te to v1 registry and other required fixes Signed-off-by: jorgee <[email protected]>

update default registry to include /api

36e9085

Signed-off-by: jorgee <[email protected]>

jorgee and others added 3 commits February 25, 2026 20:14

Merge branch '251117-module-system' into 251117-module-system-impleme…

d4418ed

…ntation

Fix compilation issue [ci skip]

fc327c4

Signed-off-by: Paolo Di Tommaso <[email protected]>

add review comments and fix compilation and tests

c3b02a8

Signed-off-by: jorgee <[email protected]>

jorgee force-pushed the 251117-module-system-implementation branch from 5aa950b to c3b02a8 Compare February 26, 2026 11:29

jorgee added 2 commits February 26, 2026 20:35

look for checksum in redirected headers

73e9707

Signed-off-by: jorgee <[email protected]>

rename checksum header

c154d35

Signed-off-by: jorgee <[email protected]>

jorgee marked this pull request as ready for review February 27, 2026 10:43

jorgee requested a review from a team as a code owner February 27, 2026 10:43

update docs

e924509

Signed-off-by: jorgee <[email protected]>

fix NPE when no nextflow.config and change preference in registry url…

412ba53

… publish (-registry -> config.url -> default) Signed-off-by: jorgee <[email protected]>

This comment was marked as outdated.

Sign in to view

This comment was marked as off-topic.

Sign in to view

jorgee and others added 4 commits February 27, 2026 14:29

update speckit files [ci skip]

119c632

Signed-off-by: jorgee <[email protected]>

Merge branch '251117-module-system' into 251117-module-system-impleme…

ad0bba2

…ntation

Merge branch '251117-module-system' into 251117-module-system-impleme…

4826533

…ntation

rename module subcommands

c20a408

Signed-off-by: jorgee <[email protected]>

pditommaso requested changes Mar 5, 2026

View reviewed changes

Comment thread docs/reference/cli.md Outdated

Comment thread docs/reference/cli.md Outdated

Comment thread docs/reference/cli.md Outdated

Comment thread docs/cli.md Outdated

Comment thread docs/reference/cli.md Outdated

address review comments

90dc444

Signed-off-by: jorgee <[email protected]>

fix test

afddf21

Signed-off-by: jorgee <[email protected]>

jorgee mentioned this pull request Mar 6, 2026

Remote module inclusion without @ #6898

Merged

6 tasks

Remote module inclusion (#6815)

18a9e70

bentsherman requested a review from a team as a code owner March 6, 2026 15:08

bentsherman and others added 2 commits March 6, 2026 09:14

update docs

a2adf6e

Signed-off-by: Ben Sherman <[email protected]>

change module remove behaviour to avoid unexpected removes

292e4f2

Signed-off-by: jorgee <[email protected]>

christopher-hakkaart reviewed Mar 8, 2026

View reviewed changes

Comment thread docs/module.md Outdated

christopher-hakkaart reviewed Mar 8, 2026

View reviewed changes

Comment thread docs/module.md Outdated

Comment thread docs/module.md Outdated

jorgee and others added 2 commits March 9, 2026 09:30

Apply suggestions from code review [ci skip]

e96c6ef

Co-authored-by: Chris Hakkaart <[email protected]> Signed-off-by: Jorge Ejarque <[email protected]>

remove pipeline spec and modules config

0b83a2c

Signed-off-by: jorgee <[email protected]>

bentsherman merged commit 583285c into 251117-module-system Mar 10, 2026
19 of 20 checks passed

bentsherman deleted the 251117-module-system-implementation branch March 10, 2026 13:22

ewels reviewed Mar 10, 2026

View reviewed changes

Comment thread docs/cli.md

bentsherman added the lang/modules label Apr 3, 2026

Uh oh!

Conversation

jorgee commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jorgee commented Jan 30, 2026

Uh oh!

bentsherman commented Feb 2, 2026

Uh oh!

jorgee commented Feb 6, 2026

Uh oh!

pditommaso commented Feb 9, 2026

Uh oh!

jorgee commented Feb 10, 2026

Uh oh!

This comment was marked as outdated.

jorgee commented Feb 19, 2026

Uh oh!

jorgee commented Feb 27, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as off-topic.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pditommaso commented Mar 5, 2026

Module System Code Review

CRITICAL (Must Fix)

1. Logic Bug — CmdModuleSearch validation (dead code)

2. Inverted Condition — CmdModuleInfo (wrong log + potential NPE)

3. e.printStackTrace() in production code

IMPORTANT (Should Fix)

5. Checksum header name mismatch between code and tests

6. Files.list() / Files.walk() stream resource leaks

7. removeModuleEntry always returns true

8. Bundle sent as raw bytes in JSON

9. Files.readAllBytes() for checksum computation

10. Missing ModulesConfig in extensions.idx

SUGGESTIONS (Nice to Have)

Strengths

Uh oh!

pditommaso commented Mar 5, 2026

Uh oh!

jorgee commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jorgee commented Mar 6, 2026

Uh oh!

Uh oh!

christopher-hakkaart left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bentsherman commented Mar 10, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jorgee commented Jan 28, 2026 •

edited

Loading

1. Logic Bug — `CmdModuleSearch` validation (dead code)

2. Inverted Condition — `CmdModuleInfo` (wrong log + potential NPE)

3. `e.printStackTrace()` in production code

6. `Files.list()` / `Files.walk()` stream resource leaks

7. `removeModuleEntry` always returns true

9. `Files.readAllBytes()` for checksum computation

10. Missing `ModulesConfig` in `extensions.idx`

jorgee commented Mar 5, 2026 •

edited

Loading