-
Notifications
You must be signed in to change notification settings - Fork 16
feat: add files include/exclude filters #202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. ✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Summary of Changes
Hello @ReneWerner87, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request introduces new functionality to the migration tool, allowing users to precisely control which migrations are applied or skipped. By adding --include and --exclude flags, the tool now supports filtering migrations using flexible glob and regex patterns, enhancing the granularity and flexibility of the migration process.
Highlights
- New Migration Filtering Options: Introduced
--includeand--excludecommand-line flags for thefiber migratecommand, enabling selective execution or skipping of migrations. - Pattern Matching Support: Implemented robust pattern matching for migration names, supporting both glob and regular expressions for the new filtering options.
- Migration Logic Update: Updated the core
DoMigrationfunction to incorporate the new include/exclude logic, ensuring only specified migrations are processed. - New Test File: Added a dedicated test file (
cmd/internal/migrations/filter_test.go) to thoroughly validate the functionality of the new migration filtering. - Documentation Update: Updated the official documentation (
docs/guide/migrate.md) to inform users about the new--includeand--excludeoptions and their usage.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces --include and --exclude flags for filtering migrations, which is a great addition. The implementation is mostly solid, with new tests and documentation. However, I've found a correctness issue in the pattern matching logic that could cause it to fail for some valid regular expressions. My review includes a suggestion to fix this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
cmd/internal/migrations/lists_test.go (1)
18-19: Remove t.Parallel to prevent races with tests that mutate global Migrations.Tests in filter_test.go override
migrations.Migrations. Running these tests in parallel introduces a data race and flakiness.Apply this diff:
func Test_DoMigration_Verbose(t *testing.T) { - t.Parallel() + // Run serially to avoid racing with tests that mutate migrations.Migrations. dir := t.TempDir() @@ t.Run("silent", func(t *testing.T) { - t.Parallel() + // Run serially; global migrations may be swapped in other tests. @@ t.Run("verbose", func(t *testing.T) { - t.Parallel() + // Run serially; global migrations may be swapped in other tests.Also applies to: 24-24, 34-35
🧹 Nitpick comments (7)
cmd/internal/migrations/lists.go (4)
155-161: Restore cmd output with defer to avoid leaking writers on panic.If a migration panics, the writer won’t be restored. Use defer right after SetOut.
Apply this diff:
- var buf bytes.Buffer - origOut := cmd.OutOrStdout() - cmd.SetOut(io.MultiWriter(origOut, &buf)) - err := fn(cmd, cwd, curr, target) - cmd.SetOut(origOut) + var buf bytes.Buffer + origOut := cmd.OutOrStdout() + cmd.SetOut(io.MultiWriter(origOut, &buf)) + defer cmd.SetOut(origOut) + err := fn(cmd, cwd, curr, target)
104-117: Regex vs. glob detection can surprise users; consider clarifying or tightening semantics.Current behavior:
- Try glob (filepath.Match). If not matched and pattern “looks like regex” (per isRegexPattern), try regex; otherwise return false.
- Regex is a substring match by default (re.MatchString), not a full-string match.
Risks:
- Patterns like
fnF.*won’t be recognized as regex (no characters from isRegexPattern), thus won’t match as users expect.- Patterns containing
?are treated as regex-worthy; after a glob miss they’ll be interpreted as regex, changing semantics.Options:
- Anchor regex matches by default (
^...$) to reduce accidental substring matches.- Expand detection to treat common regex constructs (
.*) as regex, or introduce an explicitre:prefix to force regex interpretation.- At minimum, document these semantics in the CLI/docs.
I recommend anchoring and recognizing
.*explicitly; add tests accordingly.Proposed local change (illustrative):
- if re, err := regexp.Compile(pattern); err == nil { - return re.MatchString(name) - } + if re, err := regexp.Compile(pattern); err == nil { + return re.MatchString(name) + }Outside this hunk, either:
- Anchor only when treating as regex:
re, _ := regexp.Compile("^" + pattern + "$")- Or add a rule in isRegexPattern to also return true when strings.Contains(pattern, ".*").
119-121: Regex heuristic misses common cases like.*; either broaden detection or document.
isRegexPatternignores.and*. A typical regex such asMigrate.*Configwon’t be detected as regex and will be treated as a literal dot + glob star pattern. Consider:
- Return true when pattern contains
.*, or- Keep the heuristic but document that regex must include one of
^ $ [ ] ( ) | + ? \to be recognized.Add targeted tests to lock this in.
95-102: Tiny nit: early return micro-optimization.You can check for empty
patternsfirst or iterate with a range over indexless slice; style is fine—no change required unless you’re micro-tuning.cmd/internal/migrations/filter_test.go (1)
46-54: Regex test looks good; consider adding a case for.*without anchors.Given current detection,
fnF.*isn’t treated as regex. Add a test to pin intended behavior (glob-only or regex fallback) and avoid future surprises.docs/guide/migrate.md (1)
26-27: Clarify pattern semantics and add examples for include/exclude.As implemented, a pattern is treated as:
- Glob by default.
- Regex only if it includes at least one of
^ $ [ ] ( ) | + ? \.Add a brief note and examples to reduce confusion, and show shell-quoting.
Proposed addition (outside this hunk):
### Include/Exclude examples - Only run migrations starting with `MigrateGo` (glob): fiber migrate --include 'MigrateGo*' - Skip all session-related migrations (regex): fiber migrate --exclude '^(?:MigrateSession.*)$' Notes: - Patterns are treated as globs unless they contain regex metacharacters like ^ $ [ ] ( ) | + ? \. - Regex matches are not anchored unless you use ^ and $, so add them when you intend a full-name match. - Quote patterns to prevent your shell from expanding them.cmd/migrate.go (1)
44-46: Trim whitespace and ignore empty entries from StringSlice flags.Users often write
--include "A, B". Cobra does not trim individual items. Sanitize before using.Apply this diff to sanitize in-place within RunE:
cmd.RunE = func(cmd *cobra.Command, _ []string) error { @@ - return migrateRunE(cmd, MigrateOptions{ + // sanitize patterns + include = sanitizePatterns(include) + exclude = sanitizePatterns(exclude) + + return migrateRunE(cmd, MigrateOptions{Add this helper (outside the changed hunk):
func sanitizePatterns(in []string) []string { out := make([]string, 0, len(in)) for _, s := range in { s = strings.TrimSpace(s) if s != "" { out = append(out, s) } } return out }Optional: tweak help to hint at trimming:
- cmd.Flags().StringSliceVar(&include, "include", nil, "Comma-separated list of migrations to include. Supports glob and regex patterns") + cmd.Flags().StringSliceVar(&include, "include", nil, "Comma-separated list (whitespace trimmed) of migrations to include. Supports glob and regex patterns") - cmd.Flags().StringSliceVar(&exclude, "exclude", nil, "Comma-separated list of migrations to exclude. Supports glob and regex patterns") + cmd.Flags().StringSliceVar(&exclude, "exclude", nil, "Comma-separated list (whitespace trimmed) of migrations to exclude. Supports glob and regex patterns")
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (5)
cmd/internal/migrations/filter_test.go(1 hunks)cmd/internal/migrations/lists.go(4 hunks)cmd/internal/migrations/lists_test.go(4 hunks)cmd/migrate.go(5 hunks)docs/guide/migrate.md(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (3)
cmd/internal/migrations/filter_test.go (1)
cmd/internal/migrations/lists.go (4)
Migrations(34-81)Migration(25-29)MigrationFn(22-22)DoMigration(125-197)
cmd/internal/migrations/lists_test.go (1)
cmd/internal/migrations/lists.go (1)
DoMigration(125-197)
cmd/migrate.go (1)
cmd/internal/migrations/lists.go (1)
DoMigration(125-197)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Build (1.25.x, windows-latest)
- GitHub Check: Build (1.25.x, macos-13)
- GitHub Check: Build (1.25.x, macos-latest)
🔇 Additional comments (7)
cmd/internal/migrations/lists.go (2)
147-154: Include/Exclude gating applied in the right order (include first, then exclude).Evaluation order and use of pre-computed function name look good and are easy to reason about.
125-125: DoMigration signature ripple-check passed.
All call sites have been updated to the new eight-parameter signature:
- cmd/migrate.go: uses
(cmd, wd, migrateFrom, targetVersion, opts.SkipGoMod, opts.Verbose, opts.Include, opts.Exclude)- cmd/internal/migrations/lists_test.go: all three tests call with eight arguments
(cmd, dir, curr, target, true/true, false/true, nil, nil)- cmd/internal/migrations/filter_test.go: both tests call with eight arguments
(cmd, dir, curr, target, true, false, include/exclude, nil/nil)No remaining call sites need adjustment.
cmd/internal/migrations/filter_test.go (1)
36-44: Include-glob test reads well and validates the happy path.Good use of a simple glob and direct stdout assertion.
cmd/internal/migrations/lists_test.go (1)
29-31: Call-site updates to new DoMigration signature look correct.Passing
nil, nilpreserves previous behavior when no filters are provided.Also applies to: 39-40, 67-68, 91-92
cmd/migrate.go (3)
30-31: Local variables for include/exclude are fine.No issues.
68-70: Threading flags through MigrateOptions is clean.Good propagation of the new fields.
149-151: DoMigration call updated correctly.Matches the new signature and passes through sanitized options.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
♻️ Duplicate comments (1)
cmd/internal/helpers.go (1)
121-134: Remove regex-heuristic; always try regex after glob.The isRegexPattern heuristic is incomplete and can misclassify valid regexes (e.g., v3.Migrate.*). Always attempt regex if glob didn’t match.
-func matchPattern(name, pattern string) bool { - if ok, err := filepath.Match(pattern, name); err == nil { - if ok { - return true - } - if !isRegexPattern(pattern) { - return false - } - } - if re, err := regexp.Compile(pattern); err == nil { - return re.MatchString(name) - } - return name == pattern -} - -func isRegexPattern(p string) bool { - return strings.ContainsAny(p, "^$[]()|+?\\") -} +func matchPattern(name, pattern string) bool { + if ok, err := filepath.Match(pattern, name); err == nil && ok { + return true + } + if re, err := regexp.Compile(pattern); err == nil { + return re.MatchString(name) + } + return name == pattern +}Also applies to: 136-138
🧹 Nitpick comments (5)
cmd/internal/helpers.go (2)
37-41: Global filter state risks tight coupling; prefer scoped configuration.Consider passing include/exclude via parameters or a context/options object to ChangeFileContent to avoid global mutable state. This will improve testability and concurrency safety if multiple migrations run in parallel.
43-49: Defensively copy slices to prevent external mutation/data races.Callers might mutate the provided slices after SetFileFilters returns. Copy the slices under the lock.
func SetFileFilters(include, exclude []string) { fileFilterMu.Lock() - fileIncludePatterns = include - fileExcludePatterns = exclude + if include != nil { + fileIncludePatterns = append([]string(nil), include...) + } else { + fileIncludePatterns = nil + } + if exclude != nil { + fileExcludePatterns = append([]string(nil), exclude...) + } else { + fileExcludePatterns = nil + } fileFilterMu.Unlock() }cmd/internal/migrations/file_filter_test.go (1)
39-53: Great cases; add precedence and output assertions.
- Add a case where both include and exclude are set to assert exclude wins.
- Assert cmd output contains the expected “replaceFoo” marker when a change occurs.
t.Run("include glob", func(t *testing.T) { @@ - require.NoError(t, migrations.DoMigration(cmd, dir, curr, target, true, false, []string{"*foo.go"}, nil)) + require.NoError(t, migrations.DoMigration(cmd, dir, curr, target, true, false, []string{"*foo.go"}, nil)) + assert.Contains(t, buf.String(), "replaceFoo") @@ }) t.Run("exclude regex", func(t *testing.T) { @@ - require.NoError(t, migrations.DoMigration(cmd, dir, curr, target, true, false, nil, []string{"^bar\\.go$"})) + require.NoError(t, migrations.DoMigration(cmd, dir, curr, target, true, false, nil, []string{"^bar\\.go$"})) + assert.Contains(t, buf.String(), "replaceFoo") @@ }) + +t.Run("include+exclude precedence (exclude wins)", func(t *testing.T) { + dir := t.TempDir() + require.NoError(t, os.WriteFile(filepath.Join(dir, "foo.go"), []byte("package main\nvar foo = 1\n"), 0o600)) + require.NoError(t, os.WriteFile(filepath.Join(dir, "bar.go"), []byte("package main\nvar foo = 1\n"), 0o600)) + var buf bytes.Buffer + cmd := &cobra.Command{} + cmd.SetOut(&buf) + // Include all *.go but exclude foo.go specifically + require.NoError(t, migrations.DoMigration(cmd, dir, curr, target, true, false, []string{"*.go"}, []string{"^foo\\.go$"})) + b, err := os.ReadFile(filepath.Join(dir, "foo.go")) + require.NoError(t, err) + assert.Contains(t, string(b), "var foo = 1") // excluded + b, err = os.ReadFile(filepath.Join(dir, "bar.go")) + require.NoError(t, err) + assert.Contains(t, string(b), "var bar = 1") // included + assert.Contains(t, buf.String(), "replaceFoo") +})Also applies to: 55-69
cmd/internal/migrations/lists.go (1)
95-97: Avoid reliance on global mutable state for filters.If DoMigration could ever run concurrently, the global filter toggling becomes a contention point. Consider threading include/exclude into ChangeFileContent via an options parameter to eliminate globals.
cmd/migrate.go (1)
44-45: Tighten UX/help text; mention repeated flags usage.Cobra’s StringSlice supports comma-separated and repeated flags. Consider clarifying and adding a quick example.
-cmd.Flags().StringSliceVar(&includeFiles, "include", nil, "Comma-separated list of files to include. Supports glob and regex patterns") -cmd.Flags().StringSliceVar(&excludeFiles, "exclude", nil, "Comma-separated list of files to exclude. Supports glob and regex patterns") +cmd.Flags().StringSliceVar(&includeFiles, "include", nil, "List of files to include (repeat flag or comma-separated). Supports glob and regex patterns, matched against cwd-relative paths") +cmd.Flags().StringSliceVar(&excludeFiles, "exclude", nil, "List of files to exclude (repeat flag or comma-separated). Supports glob and regex patterns, matched against cwd-relative paths")
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (5)
cmd/internal/helpers.go(4 hunks)cmd/internal/migrations/file_filter_test.go(1 hunks)cmd/internal/migrations/lists.go(3 hunks)cmd/migrate.go(5 hunks)docs/guide/migrate.md(2 hunks)
✅ Files skipped from review due to trivial changes (1)
- docs/guide/migrate.md
🧰 Additional context used
🧬 Code graph analysis (3)
cmd/internal/migrations/file_filter_test.go (2)
cmd/internal/helpers.go (1)
ChangeFileContent(54-110)cmd/internal/migrations/lists.go (4)
Migrations(32-79)Migration(23-27)MigrationFn(20-20)DoMigration(95-162)
cmd/internal/migrations/lists.go (1)
cmd/internal/helpers.go (1)
SetFileFilters(44-49)
cmd/migrate.go (1)
cmd/internal/migrations/lists.go (1)
DoMigration(95-162)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Build (1.25.x, macos-13)
- GitHub Check: Build (1.25.x, macos-latest)
- GitHub Check: Build (1.25.x, windows-latest)
🔇 Additional comments (11)
cmd/internal/helpers.go (3)
7-9: New imports are appropriate.regex and sync are required for pattern matching and guarding shared state.
71-85: Filtering order and match target are sane; clarify semantics.Files are matched against cwd-relative paths, with include first then exclude. Please confirm docs/help text explicitly state “patterns apply to relative paths” and “exclude further narrows include.”
112-119: Pattern list matcher is clear and efficient.cmd/internal/migrations/file_filter_test.go (1)
19-27: Helper keeps test intent focused.replaceFoo composes well with ChangeFileContent and emits a marker for verification.
cmd/internal/migrations/lists.go (3)
95-97: Filters are correctly scoped to the DoMigration call.Setting and resetting via defer avoids leakage across runs. Good.
119-119: Precomputing migration name is a nice micro-optimization and improves consistency.
136-136: Consistent error labeling with migration name is helpful.cmd/migrate.go (4)
30-31: New flag variables look good.
68-69: Properly wiring include/exclude into options.
83-85: MigrateOptions extension is straightforward and non-breaking.
149-149: Plumbing to DoMigration is correct.
Summary
--includeand--excludeflagsTesting
make lintmake testhttps://chatgpt.com/codex/tasks/task_e_68add1a30a6c8326b7dd067925768aca
Summary by CodeRabbit
New Features
Documentation
Tests