Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix(p6): distinguish report-format headings from prompt extraction - #513

Open
chrisknvidia wants to merge 5 commits into
NVIDIA:mainfrom
chrisknvidia:feat/christopherk/p6-output-rules
Open

fix(p6): distinguish report-format headings from prompt extraction#513
chrisknvidia wants to merge 5 commits into
NVIDIA:mainfrom
chrisknvidia:feat/christopherk/p6-output-rules

Conversation

@chrisknvidia

@chrisknvidia chrisknvidia commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

A complete formatting label such as ## HTML Output Rules (Offline-Safe) incorrectly raises HIGH P6. This change exempts its mapped noun span while retaining instructions that use the heading as a command, including framing recovered from markup or obfuscated text.

Refs #512. Keep this PR in draft pending maintainer review and release coordination.

Heading decisions are prepared once from complete source context, then mapped into bounded raw, normalized, reconstructed and overlapping scan views before finding-budget accounting. P6 matches survive window boundaries, with opt-in whitespace and invisible-separator interpretations. Other analyzers keep their existing input semantics; P7/P8 detection patterns are unchanged. Ambiguous context, headings longer than 4,096 characters, or source/normalized context beyond 1,048,576 characters retain conservative detection. Absolute coordinates are resolved lazily to avoid allocating full per-character maps during prepared analysis.

A separate commit repairs two CLI defects found during verification: recursive JSON stdout now contains the structured report with progress on stderr, and invalid provider configuration permits static-only scans while enabled LLM scans reject it cleanly.

Validation at ec52f0f8173e0a797179f2fe75abd5793b4bef54:

  • Full local unit suite with coverage: 4,201 passed, 14 skipped, 4 xfailed. Hosted lint/format, DCO and Docker smoke checks pass.
  • Independent bug/security closure review found no remaining blocker in scope. The final regression selection passed 99 tests, including ten real JSON/SARIF graph scans; 188 focused performance-fix tests passed under coverage. These selections overlap the full suite.
  • 411-input replay / 822 direct and runner evaluations: zero errors, zero new regressions, and all 95 previously missed variants restored. Expected finding-budget limits remain disclosed; source positions and bounded evidence are preserved. All observations match the preceding functional commit after excluding timing.
  • Fresh Python 3.12 wheel installation matches all 97 Python source files. Eight installed-CLI cases pass, including recursive stdout, verbose mode, invalid-provider static scans and clean LLM rejection.
  • The real downstream evaluator accepts the benign fixture and retains errors for four malicious controls, including cross-window extraction; ten runs compare unchanged base and the installed fix on the same inputs.
  • Eight static and eight authenticated live scans preserve the P6 contract. All seven synthetic inputs complete in both modes. A reference-bearing input retains its existing unresolved-reference incompleteness and a separate prose finding.
  • Exact-head hosted CI passes all checks, including 4,201 tests under coverage and Docker scans of a local directory and a GitHub URL.

Lazy source lookup lowered the measured P6 CPU cost on the oversized-artifact fixture from 2.09 seconds to 1.60, versus 1.58 on base, with unchanged resource limits. Independent offset-equivalence checks and the hosted oversized-artifact regressions pass.

Scope: this resolves the report-heading false positive and the reviewed detection/CLI regressions. Arbitrary prose noun phrases remain outside the heading exemption. No release, merge or production adoption is included.

@chrisknvidia
chrisknvidia marked this pull request as ready for review September 10, 2026 05:01
@chrisknvidia
chrisknvidia marked this pull request as draft September 10, 2026 05:11

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Reviewed head ec52f0f8173e0a797179f2fe75abd5793b4bef54 — APPROVE.

The complete-source preparation prevents a formatting heading from being exempted on ambiguous or reconstructed command context, preserves exact source mapping across bounded views, keeps P7/P8 semantics unchanged, and repairs JSON stdout plus static-only invalid-provider isolation with coverage. I found no required code or test changes.

Required checks pass, but GitHub reports mergeStateStatus=BEHIND. The PR body also explicitly says to keep the change pending maintainer review and release coordination; treat that as a hold even after updating and re-running checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants