Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@ryan-m-walker
Copy link
Contributor

Summary

Issue

Adds multi-file testing support for lint rule documentation. Rules that need to analyze relationships between multiple files (like import cycles) can now use file=path annotations to create realistic test scenarios.

  • New file=path syntax: Code blocks can specify file paths to build an in-memory file system
  • Section-based file collection: All files in a documentation section are collected before running tests, so each test sees the complete file system regardless of definition order
  • Module graph integration: Creates proper module graphs for rules that need cross-file dependency analysis
  • Backward compatible: Existing single-file tests work exactly as before

Example usage:

/// **`foobar.js`**
/// ```js,file=foobar.js,expect_diagnostic
///  import { baz } from "./baz.js";
///
/// export function foo() {
///     baz();
/// }
///
/// export function bar() {
///     console.log("foobar");
/// }
/// ```
///
/// **`baz.js`**
/// ```js,file=baz.js,expect_diagnostic
/// import { bar } from "./foobar.js";
///
/// export function baz() {
///     bar();
/// }
/// ```

Test Plan

cargo run -p rules_check

Testing was done manually using the no_import_cycles documentation by modifying the code blocks (for example removing an import from an invalid block will make the rule check fail or removing the type imports from valid blocks will cause a lint error)

Example of changing an invalid code block to now be valid and not report a diagnostic

Screenshot 2025-09-04 at 5 09 00 PM

Example of changing a valid code block to no longer be valid and report a diagnostic

Screenshot 2025-09-04 at 5 12 39 PM

Docs

crates/biome_analyze/CONTRIBUTING.md was updated so that lint rule contributors know how to utilize this feature

@changeset-bot
Copy link

changeset-bot bot commented Sep 5, 2025

⚠️ No Changeset found

Latest commit: 2f66588

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 5, 2025

Walkthrough

Documentation and test tooling were extended to support multi-file code snippets via a file=<path> code-fence property. CONTRIBUTING.md now documents multi-file snippets and ordering rules; lint rule examples (no_import_cycles) were annotated with file and diagnostic metadata. xtask/rules_check added three workspace dependencies and was reworked to collect section-scoped tests, add file_path to CodeBlockTest, build an in-memory file system and module graph from annotated files, and batch/flush tests on heading boundaries. No public APIs or runtime behaviour were changed.

Suggested labels

A-Project

Suggested reviewers

  • arendjr
  • siketyan
  • dyc3
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added A-Linter Area: linter A-Tooling Area: internal tools L-JavaScript Language: JavaScript and super languages labels Sep 5, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
xtask/rules_check/src/lib.rs (2)

546-552: Bug: wrong language used for CSS analyzer options.

create_analyzer_options::<JsonLanguage> should be ::<CssLanguage> here; wrong settings may skew diagnostics.

Apply this diff:

-                let options = create_analyzer_options::<JsonLanguage>(
+                let options = create_analyzer_options::<CssLanguage>(
                     &workspace_settings,
                     project_key,
                     &file_path,
                     test,
                 );

592-599: Bug: wrong language used for GraphQL analyzer options.

Same issue: use ::<GraphqlLanguage>.

Apply this diff:

-                let options = create_analyzer_options::<JsonLanguage>(
+                let options = create_analyzer_options::<GraphqlLanguage>(
                     &workspace_settings,
                     project_key,
                     &file_path,
                     test,
                 );
🧹 Nitpick comments (3)
crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs (1)

196-201: Nicer path display in diagnostics (strip leading slash for in‑memory paths).

When running under the in‑memory FS, users may see absolute “/foo.js”. Consider stripping a single leading “/” if cwd isn’t a prefix.

Apply this diff:

-            match Utf8Path::new(path).strip_prefix(&cwd) {
-                Ok(relative_path) => {
-                    note.extend_with(markup!(<Info>{relative_path.as_str()}</Info>))
-                }
-                Err(_) => note.extend_with(markup!(<Info>{path}</Info>)),
-            }
+            match Utf8Path::new(path).strip_prefix(&cwd) {
+                Ok(relative_path) => {
+                    note.extend_with(markup!(<Info>{relative_path.as_str()}</Info>))
+                }
+                Err(_) => {
+                    let display = path.strip_prefix('/').unwrap_or(path);
+                    note.extend_with(markup!(<Info>{display}</Info>))
+                }
+            }
xtask/rules_check/src/lib.rs (2)

221-239: Parser for file=...: OK, but consider normalising more robustly.

The trim chain works, but it only removes a single leading ../. If you ever need to support nested dirs like ../../a/b.ts, a loop or a path normaliser would be sturdier. Low-priority.


453-460: Avoid rebuilding the module graph for every test in a section.

You rebuild the graph per test even though the file set doesn’t change within the section. Build once per section and reuse; this will speed up large docs.

Apply these diffs to plumb a single graph through:

- fn assert_lint(
+ fn assert_lint(
     group: &'static str,
     rule: &'static str,
     test: &CodeBlockTest,
     code: &str,
     config: &Option<Configuration>,
-    test_files: &HashMap<String, String>,
+    test_files: &HashMap<String, String>,
+    module_graph: &Arc<ModuleGraph>,
 ) -> anyhow::Result<()> {
@@
-                let module_graph = get_test_module_graph(test_files);
-
-                let services = JsAnalyzerServices::from((
-                    Arc::new(module_graph),
+                let services = JsAnalyzerServices::from((
+                    Arc::clone(module_graph),
                     Default::default(),
                     file_source,
                 ));

and in TestRunner::run_pending_tests:

     pub fn run_pending_tests(&mut self) -> anyhow::Result<()> {
-        for test in &self.pending_tests {
+        let mg = Arc::new(get_test_module_graph(&self.file_system));
+        for test in &self.pending_tests {
             assert_lint(
                 self.group,
                 self.rule_name,
                 &test.test,
                 &test.block,
                 &test.options_snapshot,
-                &self.file_system,
+                &self.file_system,
+                &mg,
             )?;
         }
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 0f38ea6 and 4f1133d.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock and included by **
📒 Files selected for processing (4)
  • crates/biome_analyze/CONTRIBUTING.md (1 hunks)
  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs (6 hunks)
  • xtask/rules_check/Cargo.toml (1 hunks)
  • xtask/rules_check/src/lib.rs (8 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{rs,toml}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Format Rust and TOML files before committing (use just f/just format).

Files:

  • xtask/rules_check/Cargo.toml
  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
  • xtask/rules_check/src/lib.rs
crates/biome_*/**

📄 CodeRabbit inference engine (CLAUDE.md)

Place core crates under /crates/biome_*/

Files:

  • crates/biome_analyze/CONTRIBUTING.md
  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
crates/biome_*_{syntax,parser,formatter,analyze,factory,semantic}/**

📄 CodeRabbit inference engine (CLAUDE.md)

Maintain the per-language crate structure: biome_{lang}_{syntax,parser,formatter,analyze,factory,semantic}

Files:

  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
🧠 Learnings (18)
📓 Common learnings
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/biome_js_analyze/lib/src/lint/nursery/*.rs : For new JavaScript lint rules generated by `just new-js-lintrule`, implement the rule in the generated file under `biome_js_analyze/lib/src/lint/nursery/`
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_service/CONTRIBUTING.md:0-0
Timestamp: 2025-08-11T11:53:15.299Z
Learning: Use the `noImportCycles` rule to observe project layout and module graph state during debugging
📚 Learning: 2025-08-11T11:40:38.097Z
Learnt from: CR
PR: biomejs/biome#0
File: CONTRIBUTING.md:0-0
Timestamp: 2025-08-11T11:40:38.097Z
Learning: Applies to crates/**/Cargo.toml : In internal crates, use `workspace = true` for internal dependencies.

Applied to files:

  • xtask/rules_check/Cargo.toml
📚 Learning: 2025-08-11T11:40:38.097Z
Learnt from: CR
PR: biomejs/biome#0
File: CONTRIBUTING.md:0-0
Timestamp: 2025-08-11T11:40:38.097Z
Learning: Applies to Cargo.toml : Centralize shared dependencies in the root Cargo.toml using workspace dependencies.

Applied to files:

  • xtask/rules_check/Cargo.toml
📚 Learning: 2025-08-11T11:48:27.774Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_formatter/CONTRIBUTING.md:0-0
Timestamp: 2025-08-11T11:48:27.774Z
Learning: Applies to crates/biome_formatter/biome_html_formatter/Cargo.toml : Add the specified dev-dependencies (biome_formatter_test, biome_html_factory, biome_html_parser, biome_parser, biome_service, countme with feature enable, iai 0.1.1, quickcheck, quickcheck_macros, tests_macros) to Cargo.toml under [dev-dependencies]

Applied to files:

  • xtask/rules_check/Cargo.toml
📚 Learning: 2025-08-11T11:48:52.001Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_js_formatter/CONTRIBUTING.md:0-0
Timestamp: 2025-08-11T11:48:52.001Z
Learning: Applies to crates/biome_js_formatter/**/Cargo.toml : Add biome_js_formatter as a path dependency in Cargo.toml: biome_js_formatter = { version = "0.0.1", path = "../biome_js_formatter" }

Applied to files:

  • xtask/rules_check/Cargo.toml
📚 Learning: 2025-08-11T11:53:15.299Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_service/CONTRIBUTING.md:0-0
Timestamp: 2025-08-11T11:53:15.299Z
Learning: Applies to crates/biome_service/src/workspace.rs : Implement the Workspace trait in src/workspace.rs

Applied to files:

  • xtask/rules_check/Cargo.toml
📚 Learning: 2025-08-11T11:53:15.299Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_service/CONTRIBUTING.md:0-0
Timestamp: 2025-08-11T11:53:15.299Z
Learning: Applies to crates/biome_service/src/workspace/watcher.tests.rs : Place watcher tests for workspace methods in src/workspace/watcher.tests.rs

Applied to files:

  • xtask/rules_check/Cargo.toml
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/**/lib/src/**/nursery/**/*.rs : Document rules with a one-line first paragraph, `## Examples` with `### Invalid` (first) and `### Valid`, and document options under `## Options` with properly marked code blocks

Applied to files:

  • crates/biome_analyze/CONTRIBUTING.md
  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/**/lib/src/**/nursery/**/*.rs : Mark code blocks in docs with a language and required properties: use `expect_diagnostic` for invalid examples (exactly one diagnostic), `options`/`full_options` for config blocks, and `use_options` for subsequent examples

Applied to files:

  • crates/biome_analyze/CONTRIBUTING.md
  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/biome_js_analyze/lib/src/lint/nursery/*.rs : For new JavaScript lint rules generated by `just new-js-lintrule`, implement the rule in the generated file under `biome_js_analyze/lib/src/lint/nursery/`

Applied to files:

  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/**/lib/src/**/nursery/**/*.rs : When porting from other linters, declare `sources: &[RuleSource::<...>]` in `declare_lint_rule!` using `.same()` or `.inspired()` as appropriate

Applied to files:

  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/**/lib/src/**/nursery/**/*.rs : In `declare_lint_rule!` declarations, set `version: "next"`

Applied to files:

  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/**/lib/src/**/nursery/**/*.rs : Use `domains` in `declare_lint_rule!` when applicable; recommended rules with domains enable only when the domain is active

Applied to files:

  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/**/lib/src/**/nursery/**/*.rs : When banning globals (e.g., `console.log`), consult the semantic model to avoid false positives on locally shadowed identifiers

Applied to files:

  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/**/lib/src/**/nursery/**/*.rs : Place all new rules in the nursery group (implement new rule files under a `nursery` directory)

Applied to files:

  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/**/lib/src/**/nursery/**/*.rs : If a rule is deprecated, set the `deprecated:` field in `declare_lint_rule!` with a reason and alternative

Applied to files:

  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/**/lib/src/**/nursery/**/*.rs : If a rule provides a code action, add `fix_kind` in `declare_lint_rule!` and use `ctx.action_category(ctx.category(), ctx.group())` and `ctx.metadata().applicability()` when constructing actions

Applied to files:

  • crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs
📚 Learning: 2025-08-17T08:55:30.118Z
Learnt from: CR
PR: biomejs/biome#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-17T08:55:30.118Z
Learning: Applies to crates/biome_*_{syntax,parser,formatter,analyze,factory,semantic}/** : Maintain the per-language crate structure: biome_{lang}_{syntax,parser,formatter,analyze,factory,semantic}

Applied to files:

  • xtask/rules_check/src/lib.rs
🧬 Code graph analysis (1)
xtask/rules_check/src/lib.rs (2)
crates/biome_test_utils/src/lib.rs (1)
  • get_added_paths (196-217)
crates/biome_service/src/workspace/server.rs (3)
  • new (119-141)
  • fs (1606-1608)
  • fs (1688-1690)
🪛 LanguageTool
crates/biome_analyze/CONTRIBUTING.md

[uncategorized] ~1214-~1214: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...are organized by documentation section (markdown headings), where all files in a section...

(MARKDOWN_NNP)

🔇 Additional comments (9)
xtask/rules_check/Cargo.toml (1)

29-33: Workspace deps: LGTM.

Adding biome_module_graph, biome_project_layout, and biome_test_utils as { workspace = true } aligns with our workspace policy and keeps versions centralised. Nothing else to nitpick here.

crates/biome_js_analyze/src/lint/nursery/no_import_cycles.rs (1)

38-39: Docs: multi-file snippets are correctly annotated.

The file=<path> + expect_diagnostic usage matches the new CONTRIBUTING guidance and should exercise the module-graph path. Nice.

If you want a quick sanity check, run cargo run -p rules_check to ensure each invalid fence emits exactly one diagnostic under the new harness.

Also applies to: 51-51, 62-62, 71-71, 78-78, 87-87, 96-96

crates/biome_analyze/CONTRIBUTING.md (1)

1210-1235: Clear, actionable guidance for multi-file snippets.

The new section reads well, examples are consistent with the harness, and the ordering rule includes file=path. Nicely done.

Also applies to: 1243-1251

xtask/rules_check/src/lib.rs (6)

177-179: New file_path hook in CodeBlockTest: looks good.

The field is optional and non-breaking; nice extension point for multi-file tests.


378-384: Default file_path fallback: sensible.

code-block.{tag} keeps behaviour for single-file tests unchanged. Good.


896-905: Section flush on heading start: good boundary choice.

Flushing on H1–H4 works well with our doc structure and guarantees complete FS context per section.


911-912: Final flush: good catch.

Ensures trailing section tests run even if the doc ends without another heading.


916-924: PendingTest snapshotting of options: solid.

Capturing options per test avoids cross-contamination. Nice.


926-943: In-memory FS per section: exactly what multi-file rules need.

Pragmatic and simple. No notes.

Comment on lines +974 to +998
/// Creates an in-memory module graph for the given files.
/// Returns an empty module graph if no files are provided.
fn get_test_module_graph(files: &HashMap<String, String>) -> ModuleGraph {
let module_graph = ModuleGraph::default();

if files.is_empty() {
return module_graph;
}

let fs = MemoryFileSystem::default();
let layout = ProjectLayout::default();

let mut added_paths = Vec::with_capacity(files.len());

for (path, src) in files.iter() {
let path_buf = Utf8PathBuf::from(path);
added_paths.push(BiomePath::new(&path_buf));
fs.insert(path_buf, src.as_bytes().to_vec());
}

let added_paths = get_added_paths(&fs, &added_paths);
module_graph.update_graph_for_js_paths(&fs, &layout, &added_paths, &[]);

module_graph
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Compile error: ModuleGraph must be mutable for update_graph_for_js_paths.

You call a mutating method on an immutable binding. This won’t compile.

Apply this diff:

-fn get_test_module_graph(files: &HashMap<String, String>) -> ModuleGraph {
-    let module_graph = ModuleGraph::default();
+fn get_test_module_graph(files: &HashMap<String, String>) -> ModuleGraph {
+    let mut module_graph = ModuleGraph::default();
@@
-    module_graph.update_graph_for_js_paths(&fs, &layout, &added_paths, &[]);
+    module_graph.update_graph_for_js_paths(&fs, &layout, &added_paths, &[]);
 
     module_graph
 }

🏁 Script executed:

#!/usr/bin/env bash
# Verify the two language generic fixes and the mutable graph spot.
rg -nC1 "create_analyzer_options::<JsonLanguage>" xtask/rules_check/src/lib.rs
rg -n "update_graph_for_js_paths" xtask/rules_check/src/lib.rs

Length of output: 605


Make module_graph mutable in get_test_module_graph
Change the binding in xtask/rules_check/src/lib.rs around line 974:

-    let module_graph = ModuleGraph::default();
+    let mut module_graph = ModuleGraph::default();
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/// Creates an in-memory module graph for the given files.
/// Returns an empty module graph if no files are provided.
fn get_test_module_graph(files: &HashMap<String, String>) -> ModuleGraph {
let module_graph = ModuleGraph::default();
if files.is_empty() {
return module_graph;
}
let fs = MemoryFileSystem::default();
let layout = ProjectLayout::default();
let mut added_paths = Vec::with_capacity(files.len());
for (path, src) in files.iter() {
let path_buf = Utf8PathBuf::from(path);
added_paths.push(BiomePath::new(&path_buf));
fs.insert(path_buf, src.as_bytes().to_vec());
}
let added_paths = get_added_paths(&fs, &added_paths);
module_graph.update_graph_for_js_paths(&fs, &layout, &added_paths, &[]);
module_graph
}
fn get_test_module_graph(files: &HashMap<String, String>) -> ModuleGraph {
let mut module_graph = ModuleGraph::default();
if files.is_empty() {
return module_graph;
}
let fs = MemoryFileSystem::default();
let layout = ProjectLayout::default();
let mut added_paths = Vec::with_capacity(files.len());
for (path, src) in files.iter() {
let path_buf = Utf8PathBuf::from(path);
added_paths.push(BiomePath::new(&path_buf));
fs.insert(path_buf, src.as_bytes().to_vec());
}
let added_paths = get_added_paths(&fs, &added_paths);
module_graph.update_graph_for_js_paths(&fs, &layout, &added_paths, &[]);
module_graph
}
🤖 Prompt for AI Agents
In xtask/rules_check/src/lib.rs around lines 974 to 998, the local variable
module_graph is declared immutable but is mutated by calling
module_graph.update_graph_for_js_paths(...); change the binding from let
module_graph = ModuleGraph::default(); to a mutable binding (let mut
module_graph = ModuleGraph::default();) so the subsequent
update_graph_for_js_paths call can modify it, leaving the rest of the function
and the final return unchanged.

/// **`foobar.js`**
/// ```js
/// import { baz } from "./baz.js";
/// ```js,expect_diagnostic,file=foobar.js
Copy link
Member

@siketyan siketyan Sep 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could read this file tag in our codegen in website to show the filename on the snippet. Then we also can remove the filename header above. This can be a follow-up improvement :)

e.g.:

Image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that sounds like a cool idea. will definitely look into it

Copy link
Member

@siketyan siketyan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

@codspeed-hq
Copy link

codspeed-hq bot commented Sep 5, 2025

CodSpeed Performance Report

Merging #7399 will not alter performance

Comparing ryan-m-walker:chore/add-multi-file-lint-rule-checking (2f66588) with main (7f53274)1

Summary

✅ 133 untouched benchmarks

Footnotes

  1. No successful run was found on main (0f38ea6) during the generation of this report, so 7f53274 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@ryan-m-walker ryan-m-walker changed the title chore(rules_check) add multi file lint rule checking feat(rules_check) add multi file lint rule checking Sep 5, 2025
@ryan-m-walker ryan-m-walker changed the title feat(rules_check) add multi file lint rule checking feat(rules_check): add multi file lint rule checking Sep 5, 2025
@ryan-m-walker ryan-m-walker changed the title feat(rules_check): add multi file lint rule checking chore(rules_check): add multi file lint rule checking Sep 5, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
xtask/rules_check/src/lib.rs (2)

591-597: Use GraphqlLanguage in the GraphQL branch
In xtask/rules_check/src/lib.rs (DocumentFileSource::Graphql case), swap JsonLanguage for GraphqlLanguage when calling create_analyzer_options:

- let options = create_analyzer_options::<JsonLanguage>(
+ let options = create_analyzer_options::<GraphqlLanguage>(
    &workspace_settings,
    project_key,
    &file_path,
    test,
);

545-551: Use CssLanguage when resolving analyzer options for CSS
In both DocumentFileSource::Css match arms (around lines 545 and 591 in xtask/rules_check/src/lib.rs), replace

-create_analyzer_options::<JsonLanguage>(
+create_analyzer_options::<CssLanguage>(
♻️ Duplicate comments (1)
xtask/rules_check/src/lib.rs (1)

973-997: Compilation fix: make module_graph mutable before update_graph_for_js_paths

This method mutates the graph; current binding won’t compile.

-fn get_test_module_graph(files: &HashMap<String, String>) -> ModuleGraph {
-    let module_graph = ModuleGraph::default();
+fn get_test_module_graph(files: &HashMap<String, String>) -> ModuleGraph {
+    let mut module_graph = ModuleGraph::default();
@@
-    module_graph.update_graph_for_js_paths(&fs, &layout, &added_paths, &[]);
+    module_graph.update_graph_for_js_paths(&fs, &layout, &added_paths, &[]);
 
     module_graph
 }
#!/usr/bin/env bash
# Spot the mutability issue and the call site
rg -n 'get_test_module_graph\(|update_graph_for_js_paths\(' xtask/rules_check/src/lib.rs
🧹 Nitpick comments (4)
xtask/rules_check/src/lib.rs (4)

221-237: Handle quoted file= values and preserve absolute paths

Support file="src/a.ts" and file='src/a.ts', and avoid over-zealous prefix trimming. Keep a leading slash if the user provides one.

-            if let Some(file) = token.strip_prefix("file=") {
-
-                if file.is_empty() {
-                    bail!("The 'file' attribute must be followed by a file path");
-                }
-
-                // Normalize to absolute paths for consistent module resolution
-                let path = file
-                    .trim_start_matches('/')
-                    .trim_start_matches("./")
-                    .trim_start_matches("../")
-                    .trim();
-                test.file_path = Some(format!("/{}", path));
-
-                continue;
-            }
+            if let Some(file) = token.strip_prefix("file=") {
+                // Allow quoted values: file="src/a.ts" or file='src/a.ts'
+                let file = file.trim().trim_matches('"').trim_matches('\'');
+                if file.is_empty() {
+                    bail!("The 'file' attribute must be followed by a file path");
+                }
+                // Normalise to an absolute path without collapsing '..'
+                let path = if file.starts_with('/') {
+                    file.to_string()
+                } else {
+                    format!("/{}", file.trim_start_matches("./"))
+                };
+                test.file_path = Some(path);
+                continue;
+            }

871-875: Detect duplicate file paths within a section

Silently overwriting files can mask mistakes. Consider warning if a file path is defined twice with different contents. Low-touch safety net.

-                        test_runner
-                            .file_system
-                            .insert(file_path.clone(), block.clone());
+                        if let Some(prev) = test_runner.file_system.insert(file_path.clone(), block.clone()) {
+                            if prev != block {
+                                eprintln!("Warning: duplicate file path '{}' in this section; last definition wins.", file_path);
+                            }
+                        }

895-904: Flush on all heading levels, not just H1–H4

H5/H6 are rare in docs, but easy to support and keeps the behaviour predictable.

-                if matches!(
-                    level,
-                    HeadingLevel::H1 | HeadingLevel::H2 | HeadingLevel::H3 | HeadingLevel::H4
-                ) {
+                if matches!(level, HeadingLevel::H1 | HeadingLevel::H2 | HeadingLevel::H3 | HeadingLevel::H4 | HeadingLevel::H5 | HeadingLevel::H6) {

413-427: Prefer file-type from the file path when provided

If a code block sets file=foo.ts but the tag is js, the TS extension should win. This avoids mis-parsing and wrong semantics.

             let (code, file_source) = match file_source.as_embedding_kind() {
@@
                 _ => (code, file_source),
             };
+            // Prefer extension-derived source when a file path is provided
+            let file_source = if let Some(fp) = &test.file_path {
+                JsFileSource::try_from(Utf8PathBuf::from(fp).as_path()).unwrap_or(file_source)
+            } else {
+                file_source
+            };
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 4f1133d and 5f11215.

📒 Files selected for processing (1)
  • xtask/rules_check/src/lib.rs (8 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{rs,toml}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Format Rust and TOML files before committing (use just f/just format).

Files:

  • xtask/rules_check/src/lib.rs
🧠 Learnings (3)
📓 Common learnings
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/biome_js_analyze/lib/src/lint/nursery/*.rs : For new JavaScript lint rules generated by `just new-js-lintrule`, implement the rule in the generated file under `biome_js_analyze/lib/src/lint/nursery/`
📚 Learning: 2025-08-17T08:56:30.831Z
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/biome_js_analyze/lib/src/lint/nursery/*.rs : For new JavaScript lint rules generated by `just new-js-lintrule`, implement the rule in the generated file under `biome_js_analyze/lib/src/lint/nursery/`

Applied to files:

  • xtask/rules_check/src/lib.rs
📚 Learning: 2025-08-17T08:55:30.118Z
Learnt from: CR
PR: biomejs/biome#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-17T08:55:30.118Z
Learning: Applies to crates/biome_*_{syntax,parser,formatter,analyze,factory,semantic}/** : Maintain the per-language crate structure: biome_{lang}_{syntax,parser,formatter,analyze,factory,semantic}

Applied to files:

  • xtask/rules_check/src/lib.rs
🧬 Code graph analysis (1)
xtask/rules_check/src/lib.rs (1)
crates/biome_test_utils/src/lib.rs (1)
  • get_added_paths (196-217)
🔇 Additional comments (3)
xtask/rules_check/src/lib.rs (3)

925-971: New TestRunner batching looks good

Section-scoped batching and the in-memory FS are a clear win for cross-file rules. Nice and tidy.


452-459: Cache module graph per section instead of per test
get_test_module_graph is called for every example in TestRunner::run_pending_tests(); move it outside the per-test loop and pass the reused graph into assert_lint to avoid redundant builds.


982-995: Drop parse-assert concern
Inspected get_added_paths in crates/biome_test_utils/src/lib.rs—there’s no assert!(diagnostics.is_empty) present, so no panic risk on unparseable files.

Likely an incorrect or invalid review comment.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
xtask/rules_check/src/lib.rs (2)

589-596: Use GraphqlLanguage for GraphQL analysis

  • xtask/rules_check/src/lib.rs:497, 543, 589 – in each DocumentFileSource::Graphql arm, replace
    create_analyzer_options::<JsonLanguage>(…)
    with
    create_analyzer_options::<GraphqlLanguage>(…)

543-549: Replace JsonLanguage with CssLanguage in both CSS branches
Both occurrences of create_analyzer_options::<JsonLanguage> inside the DocumentFileSource::Css arm must be updated to create_analyzer_options::<CssLanguage>:

  • xtask/rules_check/src/lib.rs line 543
  • xtask/rules_check/src/lib.rs line 589
♻️ Duplicate comments (1)
xtask/rules_check/src/lib.rs (1)

974-995: Compilation fix: make ModuleGraph mutable

update_graph_for_js_paths mutates the graph; the binding must be mutable. This is already flagged earlier; still outstanding.

Apply:

-    let module_graph = ModuleGraph::default();
+    let mut module_graph = ModuleGraph::default();

Sanity check:

#!/usr/bin/env bash
rg -n 'let\s+mut\s+module_graph\s*=\s*ModuleGraph::default' xtask/rules_check/src/lib.rs
rg -n 'update_graph_for_js_paths' xtask/rules_check/src/lib.rs
🧹 Nitpick comments (3)
xtask/rules_check/src/lib.rs (3)

221-236: Avoid accidental '///foo.js' and accept absolute paths in file=…

If a user writes file=/foo.js, we currently produce ///foo.js. Let’s respect absolute inputs and only prefix when needed.

-                // Normalize to absolute paths for consistent module resolution
-                let path = file
-                    .trim_start_matches("./")
-                    .trim_start_matches("../")
-                    .trim();
-                test.file_path = Some(format!("/{}", path));
+                // Normalise to an absolute path for consistent module resolution
+                let raw = file.trim();
+                let normalised = raw
+                    .trim_start_matches("./")
+                    .trim_start_matches("../");
+                let abs = if normalised.starts_with('/') {
+                    normalised.to_string()
+                } else {
+                    format!("/{}", normalised)
+                };
+                test.file_path = Some(abs);

951-962: Build the module graph once per section to speed up tests

get_test_module_graph is called for every test, but self.file_system is identical within a section. Precompute once in run_pending_tests and reuse.

Sketch:

  • In run_pending_tests, before the loop:
let module_graph = Arc::new(get_test_module_graph(&self.file_system));
  • Thread &module_graph into assert_lint (add an optional module_graph: Option<&Arc<ModuleGraph>> param).
  • In the JS branch of assert_lint, use the provided graph (clone the Arc) instead of rebuilding.

Happy to draft a full patch if you want this in the same PR.

Also applies to: 450-456


893-902: Consider flushing on H5/H6 as well

Minor, but this future-proofs section detection if docs ever use deeper headings.

-                if matches!(
-                    level,
-                    HeadingLevel::H1 | HeadingLevel::H2 | HeadingLevel::H3 | HeadingLevel::H4
-                ) {
+                if matches!(
+                    level,
+                    HeadingLevel::H1
+                        | HeadingLevel::H2
+                        | HeadingLevel::H3
+                        | HeadingLevel::H4
+                        | HeadingLevel::H5
+                        | HeadingLevel::H6
+                ) {
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 5f11215 and 2f66588.

📒 Files selected for processing (2)
  • crates/biome_analyze/CONTRIBUTING.md (1 hunks)
  • xtask/rules_check/src/lib.rs (8 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/biome_analyze/CONTRIBUTING.md
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{rs,toml}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Format Rust and TOML files before committing (use just f/just format).

Files:

  • xtask/rules_check/src/lib.rs
🧠 Learnings (2)
📓 Common learnings
Learnt from: CR
PR: biomejs/biome#0
File: crates/biome_analyze/CONTRIBUTING.md:0-0
Timestamp: 2025-08-17T08:56:30.831Z
Learning: Applies to crates/biome_analyze/crates/biome_js_analyze/lib/src/lint/nursery/*.rs : For new JavaScript lint rules generated by `just new-js-lintrule`, implement the rule in the generated file under `biome_js_analyze/lib/src/lint/nursery/`
📚 Learning: 2025-08-17T08:55:30.118Z
Learnt from: CR
PR: biomejs/biome#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-17T08:55:30.118Z
Learning: Applies to crates/biome_*_{syntax,parser,formatter,analyze,factory,semantic}/** : Maintain the per-language crate structure: biome_{lang}_{syntax,parser,formatter,analyze,factory,semantic}

Applied to files:

  • xtask/rules_check/src/lib.rs
🧬 Code graph analysis (1)
xtask/rules_check/src/lib.rs (2)
crates/biome_test_utils/src/lib.rs (1)
  • get_added_paths (196-217)
crates/biome_lsp/src/server.rs (1)
  • default (552-554)
🔇 Additional comments (1)
xtask/rules_check/src/lib.rs (1)

941-969: Nice batching design

The section-scoped batching and FS reset are clean and make multi-file tests deterministic. Good stuff.

Copy link
Contributor

@arendjr arendjr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice work! Thanks for updating the CONTRIBUTING.md too.

kedevked pushed a commit to kedevked/biome that referenced this pull request Sep 22, 2025
dyc3 pushed a commit that referenced this pull request Oct 3, 2025
@coderabbitai coderabbitai bot mentioned this pull request Oct 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Linter Area: linter A-Tooling Area: internal tools L-JavaScript Language: JavaScript and super languages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants