Bug report

Expected behavior and actual behavior

Expected: When nextflow plugin install <plugin>@<version> is run and the registry returns HTTP 429 (rate limit), Nextflow should retry with backoff and/or degrade gracefully — it should not abort startup. Additionally, if all required plugins are already installed locally, the metadata prefetch network call should be skipped entirely.

Actual: HttpPluginRepository.sendAndParse() throws a hard PluginRuntimeException on any non-200 response including 429, crashing Nextflow startup unconditionally. With ~40 parallel CI jobs, registry.nextflow.io/api/v1/plugins/dependencies is reliably rate-limited and every job fails at startup.

Steps to reproduce the problem

1. Set up a CI system with 40+ parallel jobs, each running:

nextflow plugin install [email protected]

2. All jobs start within a short window (typical in matrix CI)
3. Every job calls registry.nextflow.io/api/v1/plugins/dependencies at startup via HttpPluginRepository.prefetch() — even if the plugin is already cached locally
4. The registry returns 429; Nextflow aborts with PluginRuntimeException

This is a regression from 25.04.3, which used DefaultUpdateRepository (static plugins.json) instead of HttpPluginRepository and did not make per-startup /api/v1/plugins/dependencies calls. Pinning back to 25.04.3 eliminates the failures.

Program output

PluginRuntimeException: Invalid response while fetching plugin metadata from:
  https://registry.nextflow.io/api/v1/plugins/dependencies?plugins=nf-schema%402.5.1&nextflowVersion=26.04.2
- http status: 429
- response   : <rate limit response body>

Root cause (specific code pointers, v26.04.2)

The call chain with no 429 handling and no skip-if-installed logic:

1. PluginsFacade.groovy:415 — unconditional call, no if (!offline) guard:

updater.prefetchMetadata(startable)

offline is correctly read from NXF_OFFLINE at line 66 and passed to PluginUpdater at line 254, but never used to guard this call.

2. PluginUpdater.groovy:150–163 — iterates repos, calls prefetch() on each PrefetchUpdateRepository. this.offline is stored on the instance but never consulted here.

3. HttpPluginRepository.groovy:142–157 — builds and fires the HTTP request on every startup, regardless of whether the plugin is already installed:

def uri = url.resolve("v1/plugins/dependencies?plugins=...&nextflowVersion=...")
return sendAndParse(HttpRequest.newBuilder().uri(uri).GET().build())

4. HttpPluginRepository.groovy:165–167 ← THE BUG — hard-aborts on any non-200 including 429, with no Retry-After handling:

if( resp.statusCode() != 200 ) {
    throw new PluginRuntimeException("Invalid response ... http status: ${resp.statusCode()} ...")
}

Environment

• Nextflow version: 26.04.2 (regression from 25.04.3)
• Java version: 21
• Operating system: Linux (self-hosted CI runners)
• Bash version: 5.x

Additional context

• Related issue Plugin metadata prefetch fails hard even when result is unused (pre-installed plugins, exact versions) #7013 "Plugin metadata prefetch fails hard even when result is unused" — closed with suggestion to use NXF_OFFLINE=true. That workaround is incomplete: it requires plugins to be pre-cached locally and prevents any network access, making it incompatible with steps that legitimately need to install plugins.
• Related PR Make plugin metadata prefetch failure non-fatal #7019 "Make plugin metadata prefetch failure non-fatal" — addresses the hard-abort symptom but was not merged.
• Suggested fixes:
  1. In sendAndParse(): handle 429 specifically — respect Retry-After header, retry with exponential backoff, don't hard-abort.
  2. In prefetchMetadata / prefetch(): skip the network call if all requested plugins are already present locally.
  3. Defence-in-depth: add if (!offline) guard at PluginsFacade.groovy:415.