-
Notifications
You must be signed in to change notification settings - Fork 16
Ensure HTTP cache directory exists before locking #195
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. Warning Rate limit exceeded@ReneWerner87 has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 5 minutes and 21 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (2)
WalkthroughThe cache directory is now created at the start of readFromFile and writeToFile via os.MkdirAll. Duplicate directory creation after acquiring the write lock was removed. Error logging was added for mkdir failures. A new test validates basic read/write behavior of the HTTP cache. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Caller
participant HTTPCache
participant OS as OS.MkdirAll
participant FS as FileSystem
Caller->>HTTPCache: readFromFile(url)
HTTPCache->>OS: MkdirAll(cacheDir, 0o750)
alt mkdir fails
HTTPCache-->>Caller: (nil, false) [log: "httpcache: mkdir ..."]
else mkdir ok
HTTPCache->>FS: Read file
alt cache hit and not expired
HTTPCache-->>Caller: (body, true)
else miss/expired
HTTPCache-->>Caller: (nil, false)
end
end
sequenceDiagram
autonumber
participant Caller
participant HTTPCache
participant OS as OS.MkdirAll
participant FS as FileSystem
participant Lock as WriteLock
Caller->>HTTPCache: writeToFile(url, body)
HTTPCache->>OS: MkdirAll(cacheDir, 0o750)
alt mkdir fails
HTTPCache-->>Caller: return [log: "httpcache: mkdir ..."]
else mkdir ok
HTTPCache->>Lock: Acquire
HTTPCache->>FS: Write file (JSON)
HTTPCache->>Lock: Release
HTTPCache-->>Caller: return
end
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested labels
Poem
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Summary of Changes
Hello @ReneWerner87, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request addresses a potential issue where HTTP cache operations could fail if the cache directory did not exist. It ensures that the necessary cache directory is created before any read or write operations are performed, preventing "no such file" errors. Additionally, a new comprehensive test has been added to validate the read and write functionality of the HTTP cache, improving the robustness and reliability of the caching mechanism.
Highlights
- Directory Creation in Read Operations: The
readFromFilefunction now includes a check and creation of the cache directory usingos.MkdirAllat its very beginning. This ensures that the necessary directory structure is in place before any file read operations or lock acquisitions are attempted, preventing "no such file or directory" errors. - Directory Creation in Write Operations: Similarly, the
writeToFilefunction has been updated to includeos.MkdirAllat its start. This guarantees the cache directory exists before writing data or acquiring write locks, addressing potential errors related to missing directories. The previousos.MkdirAllcall later in the function was removed as it became redundant. - New HTTP Cache Test: A new test file,
cmd/httpcache_test.go, has been introduced. This file containsTest_HTTPCache_ReadWrite, which thoroughly tests the cache's read and write mechanisms, including scenarios where a file is not yet cached, then written, and subsequently read successfully.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request correctly addresses a potential 'file not found' error by ensuring the HTTP cache directory is created before attempting to acquire a file lock. The changes are logical, and the new test case is a good addition to verify the file-based caching mechanism. I have a couple of suggestions to improve the implementation. First, to avoid redundant system calls, the cache directory creation could be moved to an init() function, making it a one-time operation. Second, the new test could be made more robust by using t.Cleanup to ensure the cache is cleared after the test runs, improving test isolation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (5)
cmd/httpcache.go (3)
40-44: Tighten directory permissions to 0o700 to minimize metadata exposure.Files are written with 0o600, but the directory is 0o750, which allows group members to list hashed filenames. While filenames are hashes, listing may still leak request cadence. Prefer 0o700 for the cache dir. On Windows, mode is mostly ignored, so this remains safe and portable.
Apply this diff to both mkdir calls:
- if err := os.MkdirAll(cacheDir, 0o750); err != nil { + if err := os.MkdirAll(cacheDir, 0o700); err != nil {Also applies to: 78-82
100-102: Use temp-file + rename for atomic writes and crash-safety.An unexpected crash during
os.WriteFilecan leave a truncated file that future readers will treat as a corrupted cache entry. Writing to a temp file in the same directory andRename-ing it into place avoids partial writes.- if err := os.WriteFile(cacheFile(url), b, 0o600); err != nil { - log.Printf("httpcache: write cache file for %s: %v", url, err) - } + cf := cacheFile(url) + tmp := cf + ".tmp" + if err := os.WriteFile(tmp, b, 0o600); err != nil { + log.Printf("httpcache: write temp cache file for %s: %v", url, err) + return + } + if err := os.Rename(tmp, cf); err != nil { + // Best effort fallback for platforms where rename may not replace. + _ = os.Remove(cf) //nolint:errcheck + if err2 := os.Rename(tmp, cf); err2 != nil { + log.Printf("httpcache: replace cache file for %s: %v", url, err2) + } + }
21-27: Align in-memory cache semantics with disk TTL to avoid indefinite staleness.The in-memory
responseCachenever expires, so a long-lived process can serve stale data far beyondcacheTTL, even though file cache enforces expiry. Consider storingcacheEntryin-memory and enforcing TTL consistently.@@ var ( cacheMu sync.RWMutex - responseCache = make(map[string][]byte) + responseCache = make(map[string]cacheEntry) @@ - cacheMu.RLock() - if b, ok := responseCache[url]; ok { - cacheMu.RUnlock() - return b, http.StatusOK, nil - } - cacheMu.RUnlock() + cacheMu.RLock() + if e, ok := responseCache[url]; ok { + if time.Now().Before(e.Expiry) { + cacheMu.RUnlock() + return e.Body, http.StatusOK, nil + } + } + cacheMu.RUnlock() @@ - if b, ok := responseCache[url]; ok { - return b, http.StatusOK, nil - } + if e, ok := responseCache[url]; ok { + if time.Now().Before(e.Expiry) { + return e.Body, http.StatusOK, nil + } + delete(responseCache, url) + } @@ - if res.StatusCode == http.StatusOK { - responseCache[url] = body - writeToFile(url, body) - } + if res.StatusCode == http.StatusOK { + responseCache[url] = cacheEntry{Expiry: time.Now().Add(cacheTTL), Body: body} + writeToFile(url, body) + } @@ - responseCache = make(map[string][]byte) + responseCache = make(map[string]cacheEntry)If you’d prefer to keep the fast path, you can also add a very small TTL (e.g., 30–60s) for memory while keeping 5 min on disk.
Also applies to: 108-114, 118-121, 151-154, 161-164
cmd/httpcache_test.go (2)
10-16: Isolate cacheDir per test using t.TempDir to avoid cross-test interference.Because
cacheDiris a package-level var, parallel or future tests can step on each other. Point the cache to a unique temp dir and restore afterward.func Test_HTTPCache_ReadWrite(t *testing.T) { - clearHTTPCache() + // Isolate cache directory for this test + oldDir := cacheDir + cacheDir = t.TempDir() + t.Cleanup(func() { cacheDir = oldDir }) + clearHTTPCache()
9-23: Optional: add an expiry subtest for read path.If you want to exercise the expiry logic end-to-end, we can temporarily shrink
cacheTTLin the test and verify that a read after expiry returnsok == false. Happy to add this as a follow-up.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
cmd/httpcache.go(2 hunks)cmd/httpcache_test.go(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Build (1.25.x, macos-latest)
- GitHub Check: Build (1.25.x, windows-latest)
- GitHub Check: Build (1.25.x, macos-13)
🔇 Additional comments (3)
cmd/httpcache.go (2)
40-44: Creating the cache directory before taking the lock fixes ENOENT race. LGTM.This directly addresses the reported “no such file” failures when the parent dir didn’t exist prior to flock creation. Good placement and diagnostics.
78-82: Mirroring the mkdir precondition in the write path is correct.Early-return on mkdir failure prevents confusing downstream errors and avoids locking when we can’t proceed anyway.
cmd/httpcache_test.go (1)
9-23: Good baseline read/write test that covers the mkdir-before-lock behavior.This validates the intended flow and would have caught the original ENOENT. Nice and concise use of testify.
Summary
Testing
GOFLAGS=-mod=mod make lintGOFLAGS=-mod=mod make testhttps://chatgpt.com/codex/tasks/task_e_68ab3541e4a48326bbe8b21059af26ff
Summary by CodeRabbit
Bug Fixes
Tests