-
-
Notifications
You must be signed in to change notification settings - Fork 330
Add DB-backed project freshness score using OWASP-Bumper #5330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
👋 Hi @Nachiket-Roy! This pull request needs a peer review before it can be merged. Please request a review from a team member who is not:
Once a valid peer review is submitted, this check will pass automatically. Thank you! |
WalkthroughAdds a stored Project.freshness field and calculate_freshness(), a migration, a management command to recalc/persist freshness (invoked from daily runner), API validation/filtering for a freshness query param, serializer change to expose stored freshness, removal of legacy ProjectViewSet.filter, and updated tests. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Client
participant API as Django API (ProjectViewSet)
participant DB as Database (Project, Repo)
Note over API,DB: API request flow for filtering projects by stored freshness
Client->>API: GET /api/v1/projects/?freshness=50
API->>API: parse & validate freshness param (float, 0–100)
alt invalid
API-->>Client: 400 Bad Request (error message)
else valid
API->>DB: Query Projects with freshness__gte=50 (plus other filters)
DB-->>API: matching Project rows (includes stored freshness)
API-->>Client: 200 OK (serialized projects with read-only freshness)
end
sequenceDiagram
autonumber
participant Scheduler
participant Cmd as update_project_freshness
participant DB as Database (Project, Repo)
participant Logger
Note over Cmd,DB: Daily freshness update batch flow
Scheduler->>Cmd: call_command("update_project_freshness")
Cmd->>DB: fetch Project ids (batched)
loop per project in batch
Cmd->>DB: load project with row lock (select_for_update)
Cmd->>Cmd: freshness = project.calculate_freshness()
alt success
Cmd->>DB: save(project, update_fields=["freshness"])
else error
Cmd->>Logger: log exception for project id
end
end
Cmd-->>Scheduler: print summary (processed, errors, elapsed time)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📊 Monthly LeaderboardHi @Nachiket-Roy! Here's how you rank for December 2025:
Leaderboard based on contributions in December 2025. Keep up the great work! 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
📒 Files selected for processing (7)
website/api/views.py(1 hunks)website/management/commands/run_daily.py(1 hunks)website/management/commands/update_project_freshness.py(1 hunks)website/migrations/0264_project_freshness.py(1 hunks)website/models.py(1 hunks)website/serializers.py(1 hunks)website/tests/test_project_aggregation.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
website/models.py (1)
website/api/views.py (2)
filter(375-414)filter(844-922)
website/management/commands/update_project_freshness.py (1)
website/models.py (12)
Project(1366-1477)calculate_freshness(1401-1440)save(74-77)save(277-289)save(1442-1469)save(1577-1580)save(1770-1773)save(1888-1902)save(1993-2015)save(2814-2817)save(3346-3356)save(3654-3663)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Run Tests
- GitHub Check: docker-test
🔇 Additional comments (6)
website/management/commands/run_daily.py (1)
48-51: New daily freshness task wiring looks consistentThe new
update_project_freshnessinvocation matches the existing error-handling pattern for other daily commands and won’t break the overall job on failures. No changes needed.website/api/views.py (1)
856-869: Freshness filter validation and semantics are soundRange checking, type validation, and
freshness__gtebehavior all look correct and consistent with existing stars/forks filters.website/tests/test_project_aggregation.py (1)
16-24: Test setup now correctly relies on real freshness behaviorKeeping only the
prefetch_relatedpatch and dropping any freshness mocking is appropriate given the new DB-backed field. The updated comment still accurately describes the remaining patch.website/serializers.py (1)
129-141: Serializer exposure of freshness matches the model and requirementsUsing a read-only
DecimalField(max_digits=5, decimal_places=2)onProjectSerializercleanly reflects the storedfreshnessvalue while preventing client writes. Looks good.website/migrations/0264_project_freshness.py (1)
1-17: Migration correctly adds indexed freshness fieldThe migration cleanly adds
Project.freshnesswith the expectedDecimalFieldshape and index, aligned with the model definition and PR intent.website/management/commands/update_project_freshness.py (1)
1-40: Freshness recalculation command is correct and robustThe command correctly:
- Iterates all projects,
- Uses
project.calculate_freshness(),- Saves only the
freshnessfield,- Isolates per-project failures while continuing,
- Reports progress and timing.
This is a solid implementation for a daily batch job.
|
@coderabbitai help add tests for the changes made in this pr |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 826 🏁 Script executed: Length of output: 10233 🏁 Script executed: Length of output: 7006 @Nachiket-Roy: I'll help you create comprehensive tests for the freshness scoring feature. Here are test files covering all the changes: 1. Model Tests -
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
📒 Files selected for processing (6)
website/api/views.py(1 hunks)website/models.py(1 hunks)website/tests/test_api.py(2 hunks)website/tests/test_project_aggregation.py(3 hunks)website/tests/test_project_freshness.py(1 hunks)website/tests/test_update_project_freshness_command.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- website/models.py
🧰 Additional context used
🧬 Code graph analysis (3)
website/tests/test_project_aggregation.py (1)
website/models.py (3)
Project(1366-1479)Repo(1948-2025)calculate_freshness(1401-1442)
website/tests/test_api.py (1)
website/models.py (2)
Project(1366-1479)Repo(1948-2025)
website/tests/test_project_freshness.py (1)
website/models.py (4)
Organization(181-289)Project(1366-1479)Repo(1948-2025)calculate_freshness(1401-1442)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: auto-assign
- GitHub Check: remove_last_active_label
- GitHub Check: add_changes_requested_label
- GitHub Check: check_peer_review
- GitHub Check: remove_last_active_label
- GitHub Check: Run Tests
🔇 Additional comments (4)
website/tests/test_project_aggregation.py (1)
160-215: Excellent integration test with correct freshness calculation.The test properly validates the end-to-end freshness calculation and persistence flow. The expected value of 9.5 is correct:
- very-active (2 days ago) → active_7 = 1 → weight 1.0
- somewhat-active (20 days ago) → active_30 = 1 → weight 0.6
- old-active (60 days ago) → active_90 = 1 → weight 0.3
- archived (1 day ago) → correctly excluded
- raw_score = 1.9, normalized = (1.9/20)*100 = 9.5
website/api/views.py (1)
760-775: LGTM! Clean implementation of freshness filtering.The validation logic is consistent with the existing stars/forks parameter handling, uses appropriate >= semantics with
freshness__gte, and provides clear error messages for invalid input.website/tests/test_update_project_freshness_command.py (1)
15-136: LGTM! Comprehensive command test coverage.The test suite thoroughly exercises the management command with good coverage of:
- Batch updates across multiple projects with varying activity levels
- Graceful per-project error handling using proper mocking
- Execution time reporting
- Zero-projects edge case
The expected freshness value of 5.0 at Line 105 is correctly calculated (1 repo active 5 days ago → raw_score 1.0 → 5.0 freshness).
website/tests/test_api.py (1)
299-427: LGTM! Comprehensive API filtering test coverage.The test suite thoroughly validates freshness filtering including:
- Threshold filtering with correct >= semantics
- Input validation for negative, >100, and non-numeric values
- Decimal freshness values
- Combined filtering with other parameters (stars)
- Freshness field presence in API responses
The use of hardcoded freshness values in setUp (Lines 307-321) is appropriate for API-level testing. The actual freshness calculation logic is validated by the integration test in
test_project_aggregation.py(lines 160-215).
CLOSES : #5226
Summary
This PR introduces a persisted freshness score for projects and computes it using a Bumper-style time-decay activity model based on repository recency. Storing this value makes project freshness queryable, filterable, and consistent across the platform.
Background : In the Projects section, the Freshness filter estimates how active a repository is. However, this value was not stored in the database, which resulted in a
FieldErrorwhen applying the filter.Solution :
This PR resolves the issue by persisting a computed freshness score on the Project model and using it to back the filter.
Freshness is calculated using the same decay model as
blt-bumper, where recent activity contributes more weight than older activity:Last commit within 7 days - 1.0
Last commit within 30 days - 0.6
Last commit within 90 days - 0.3
To prevent highly active repositories from dominating the ranking, the score is capped and normalized to a 0–100 scale.
A management command is also included to populate freshness for all existing projects:
Key Changes
calculate_freshness()using:Algorithm Notes
The freshness score follows a Bumper-style decay model. Repositories are categorized into rolling 7-day, 30-day, and 90-day windows based on their most recent activity. Each window contributes with progressively lower weights (1.0, 0.6, 0.3) to reflect diminishing relevance over time.
The weighted score is normalized and capped to ensure fairness, preventing large or extremely active projects from disproportionately outweighing less active ones. This mirrors Bumper’s activity graph philosophy by prioritizing recent contributions while naturally discounting stale repositories.
Why this approach
Summary by CodeRabbit
New Features
Bug Fixes
Chores
Tests
✏️ Tip: You can customize this high-level summary in your review settings.