-
-
Notifications
You must be signed in to change notification settings - Fork 313
Fixing Issue #4220 | Added Throttling Middleware for normal views #4385
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixing Issue #4220 | Added Throttling Middleware for normal views #4385
Conversation
|
""" WalkthroughA new Django middleware, Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant ThrottlingMiddleware
participant DjangoApp
participant Cache
Client->>ThrottlingMiddleware: Sends HTTP request
ThrottlingMiddleware->>ThrottlingMiddleware: Check if path or view is exempt
alt Exempt
ThrottlingMiddleware->>DjangoApp: Pass request through
DjangoApp-->>Client: Return response
else Not exempt
ThrottlingMiddleware->>Cache: Get request count for IP/method
alt Count exceeds limit
ThrottlingMiddleware-->>Client: Return HTTP 429 with Retry-After
else Count within limit
ThrottlingMiddleware->>Cache: Increment request count
ThrottlingMiddleware->>DjangoApp: Pass request through
DjangoApp-->>Client: Return response
end
end
Assessment against linked issues
Assessment against linked issues: Out-of-scope changesNo out-of-scope changes were found. ✨ Finishing Touches
🧪 Generate Unit Tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 6
🧹 Nitpick comments (2)
blt/settings.py (1)
124-124: Consider placing ThrottlingMiddleware after authentication middlewareThe throttling middleware is currently placed before
AuthenticationMiddleware(line 119). This ordering may prevent the middleware from properly distinguishing between authenticated and anonymous users if such functionality is added in the future. Consider moving it after the authentication-related middleware but before static file serving middleware.MIDDLEWARE = ( "django.contrib.sessions.middleware.SessionMiddleware", "blt.middleware.domain.DomainMiddleware", "django.middleware.locale.LocaleMiddleware", "django.middleware.common.CommonMiddleware", "django.middleware.csrf.CsrfViewMiddleware", "django.contrib.auth.middleware.AuthenticationMiddleware", "django.contrib.messages.middleware.MessageMiddleware", "allauth.account.middleware.AccountMiddleware", "django.middleware.clickjacking.XFrameOptionsMiddleware", "django.middleware.security.SecurityMiddleware", - "blt.middleware.throttling.ThrottlingMiddleware", "whitenoise.middleware.WhiteNoiseMiddleware", + "blt.middleware.throttling.ThrottlingMiddleware", "tz_detect.middleware.TimezoneMiddleware", "blt.middleware.ip_restrict.IPRestrictMiddleware", "blt.middleware.user_visit_tracking.VisitTrackingMiddleware", )blt/middleware/throttling.py (1)
12-18: Make throttle limits configurable via Django settingsThe throttle limits and window are currently hardcoded. Consider making them configurable through Django settings for better flexibility across different environments.
+from django.conf import settings + class ThrottlingMiddleware: - THROTTLE_LIMITS = { - "GET": 100, # 100 GET requests per minute - "POST": 50, # 50 POST requests per minute - "OTHER": 30, # 30 other requests per minute - } - THROTTLE_WINDOW = 60 # 60 seconds - EXEMPT_PATHS = ["/admin/", "/static/", "/media/"] + THROTTLE_LIMITS = getattr(settings, 'THROTTLE_LIMITS', { + "GET": 100, + "POST": 50, + "OTHER": 30, + }) + THROTTLE_WINDOW = getattr(settings, 'THROTTLE_WINDOW', 60) + EXEMPT_PATHS = getattr(settings, 'THROTTLE_EXEMPT_PATHS', ["/admin/", "/static/", "/media/"])
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Knowledge Base: Disabled due to Reviews > Disable Knowledge Base setting
📒 Files selected for processing (3)
blt/middleware/throttling.py(1 hunks)blt/settings.py(1 hunks)website/tests/test_middleware.py(1 hunks)
🧰 Additional context used
🪛 Pylint (3.3.7)
website/tests/test_middleware.py
[refactor] 23-23: Too few public methods (1/2)
(R0903)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Run Tests
- GitHub Check: docker-test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
♻️ Duplicate comments (1)
website/tests/test_middleware.py (1)
33-33: Fix missing f-string prefix.The assertion message uses string interpolation syntax but lacks the required f-string prefix.
- self.assertEqual(response.status_code, 200, "Request {i+1} should be allowed") + self.assertEqual(response.status_code, 200, f"Request {i+1} should be allowed")
🧹 Nitpick comments (3)
blt/middleware/throttling.py (1)
113-124: Consider edge cases in IP extraction.The IP extraction logic is solid, but consider handling edge cases where
x_forwarded_formight be empty or contain invalid values.def get_client_ip(self, request): """Extract client IP from request.""" x_forwarded_for = request.META.get("HTTP_X_FORWARDED_FOR") - ip = x_forwarded_for.split(",")[0].strip() if x_forwarded_for else request.META.get("REMOTE_ADDR") + if x_forwarded_for: + ip = x_forwarded_for.split(",")[0].strip() + # Handle empty string after strip + if not ip: + ip = request.META.get("REMOTE_ADDR") + else: + ip = request.META.get("REMOTE_ADDR")website/tests/test_middleware.py (2)
25-38: Add validation for throttle response headers.The test verifies the 429 status code but should also validate the
Retry-Afterheader is set correctly.# 101st request should be throttled request = self.factory.get("/some-path", REMOTE_ADDR=ip) response = self.middleware(request) self.assertEqual(response.status_code, 429, "101st request should be throttled") +self.assertIn("Retry-After", response) +self.assertEqual(response["Retry-After"], "60") # Default THROTTLE_WINDOW
18-23: Consider testing cache key isolation.The tests are well-structured, but consider adding a test to verify that different HTTP methods for the same IP are tracked separately (e.g., GET vs POST requests).
Add a new test method:
def test_different_methods_tracked_separately(self): """Test that different HTTP methods for same IP are tracked separately.""" ip = "192.168.1.1" # Make 100 GET requests for i in range(100): request = self.factory.get("/some-path", REMOTE_ADDR=ip) response = self.middleware(request) self.assertEqual(response.status_code, 200) # POST requests should still be allowed (different cache key) for i in range(50): request = self.factory.post("/some-path", REMOTE_ADDR=ip) response = self.middleware(request) self.assertEqual(response.status_code, 200)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Knowledge Base: Disabled due to Reviews > Disable Knowledge Base setting
📒 Files selected for processing (3)
blt/middleware/throttling.py(1 hunks)blt/settings.py(1 hunks)website/tests/test_middleware.py(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- blt/settings.py
🧰 Additional context used
🪛 Pylint (3.3.7)
website/tests/test_middleware.py
[refactor] 13-13: Too few public methods (1/2)
(R0903)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Run Tests
- GitHub Check: docker-test
🔇 Additional comments (4)
blt/middleware/throttling.py (3)
64-64: Good fix for the DRF view detection logic.The parentheses are now correctly placed and
view_classis properly checked before callingissubclass, addressing the previous TypeError issue.
97-110: Excellent implementation of atomic cache operations.The race condition issue has been properly addressed using
get_or_setfor atomic initialization and handling theValueErrorexception case when the key expires between operations.
42-44: Correct HTTP 429 response implementation.The response now properly uses
HttpResponsewith status 429 and sets theRetry-Afterheader with just the numeric value, complying with RFC 7231.website/tests/test_middleware.py (1)
61-72: Excellent DRF view exemption test.This test properly addresses the previous comment about testing DRF view exemption. The mock resolver_match setup correctly simulates how Django REST Framework views are identified.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🧹 Nitpick comments (1)
website/test_throttling_middleware.py (1)
60-71: Simplify the DRF mock creation for better readability.The test logic is sound and effectively validates DRF view exemptions, but the inline mock creation is complex and reduces readability.
Consider extracting the mock creation to a helper method:
+ def _create_drf_request(self, path, ip): + """Helper to create a request that appears to be routed to a DRF view.""" + request = self.factory.get(path, REMOTE_ADDR=ip) + request.resolver_match = type( + "resolver", (), {"func": type("func", (), {"cls": MockDRFView, "is_api_view": True})} + )() + return request def test_drf_views_not_throttled(self): """Test that DRF views are exempt from throttling.""" ip = "192.168.1.1" # Make many requests to DRF view for i in range(150): - request = self.factory.get("/api/test/", REMOTE_ADDR=ip) - request.resolver_match = type( - "resolver", (), {"func": type("func", (), {"cls": MockDRFView, "is_api_view": True})} - )() + request = self._create_drf_request("/api/test/", ip) response = self.middleware(request) self.assertEqual(response.status_code, 200, f"DRF request {i+1} should not be throttled")Additionally, consider testing different HTTP methods beyond just GET requests to ensure comprehensive coverage of the throttling middleware.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Knowledge Base: Disabled due to Reviews > Disable Knowledge Base setting
📒 Files selected for processing (3)
blt/middleware/throttling.py(1 hunks)blt/settings.py(2 hunks)website/test_throttling_middleware.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- blt/settings.py
- blt/middleware/throttling.py
🧰 Additional context used
🪛 Pylint (3.3.7)
website/test_throttling_middleware.py
[refactor] 12-12: Too few public methods (1/2)
(R0903)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Run Tests
- GitHub Check: docker-test
🔇 Additional comments (4)
website/test_throttling_middleware.py (4)
1-9: Well-organized imports with proper dependencies.The imports are correctly structured and include all necessary testing dependencies including Django's testing framework, DRF components, and the middleware under test.
11-15: Appropriate mock class for testing DRF exemptions.The mock DRF view is simple and serves its purpose well for testing middleware exemptions. The static analysis warning about "too few public methods" can be safely ignored as this is a test mock.
17-23: Excellent test setup with proper isolation.The test setup correctly uses RequestFactory and ensures test isolation by clearing the cache before each test. The middleware initialization with a lambda response handler is appropriate for testing.
39-47: Excellent test for IP-based tracking independence.This test effectively validates that the middleware tracks different IP addresses independently, which is crucial for fair rate limiting. The approach of testing 150 different IPs ensures robust validation of the separation logic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🧹 Nitpick comments (1)
website/test_throttling_middleware.py (1)
63-74: Simplify the mock resolver match creation.The nested
type()calls create a fragile and hard-to-read mock structure. Consider using a more straightforward approach.Simplify the mock resolver_match creation:
def test_drf_views_not_throttled(self): """Test that DRF views are exempt from throttling.""" ip = "192.168.1.1" # Make many requests to DRF view for i in range(150): request = self.factory.get("/api/test/", REMOTE_ADDR=ip) - request.resolver_match = type( - "resolver", (), {"func": type("func", (), {"cls": MockDRFView, "is_api_view": True})} - )() + # Create a simple mock resolver match + class MockResolver: + def __init__(self): + self.func = MockDRFView + self.func.cls = MockDRFView + self.func.is_api_view = True + + request.resolver_match = MockResolver() response = self.middleware(request) self.assertEqual(response.status_code, 200, f"DRF request {i+1} should not be throttled")
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Knowledge Base: Disabled due to Reviews > Disable Knowledge Base setting
📒 Files selected for processing (1)
website/test_throttling_middleware.py(1 hunks)
🧰 Additional context used
🪛 Pylint (3.3.7)
website/test_throttling_middleware.py
[refactor] 13-13: Too few public methods (1/2)
(R0903)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Run Tests
- GitHub Check: docker-test
🔇 Additional comments (5)
website/test_throttling_middleware.py (5)
1-10: LGTM - Well-structured imports and dependencies.The imports are comprehensive and appropriate for testing Django middleware functionality. Good use of Django's testing framework components and REST framework imports for DRF view testing.
12-16: Mock class serves its purpose despite static analysis warning.The
MockDRFViewclass appropriately simulates a DRF APIView for testing exemption logic. The static analysis warning about "too few public methods" is a false positive - test mocks don't need multiple public methods.
18-24: Excellent test setup with proper isolation.The setup method correctly initializes all necessary components and importantly clears the cache before each test to ensure test isolation. This prevents test interdependencies that could cause flaky results.
25-41: Good improvement addressing configuration dependency.This test correctly retrieves the throttling limit from settings rather than hardcoding it, addressing the previous review feedback. The test logic properly validates both allowed and throttled scenarios.
51-62: Excellent configuration-driven approach.This test correctly addresses the previous feedback by importing exempt paths from settings rather than hardcoding them. The test comprehensively validates that exempt paths are never throttled regardless of request volume.
) * middleaware changes with tests * covered requested changes * changed test location and suggested changes * refactor --------- Co-authored-by: DonnieBLT <[email protected]>
|
Any idea why it’s not merging? |
Fixes #4220
Added Throttling Middleware for normal views, it does take care of skipping DRF views as they are already throttled.
Summary by CodeRabbit