Response improvements and refactoring #2407
Conversation
Summary by CodeRabbit
WalkthroughReplaces hard-coded system prompts with dynamic Prompt model getters, updates default retrieval constants, adds Slack Markdown-to-Slack-link formatting and integrates it into Slack blocks, and updates tests to mock/use the new prompt getters and new defaults. No public API signature removals; two new Prompt accessors and one new Slack util added. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
backend/apps/slack/utils.py (1)
51-51: Optimize regex compilation.The regex pattern is compiled on every function call. Moving it to a module-level constant would improve performance, especially if this function is called frequently.
Apply this diff to optimize:
+# Module-level constant for better performance +_MARKDOWN_LINK_PATTERN = re.compile(r"\[([^\]]+)\]\((https?://[^\s)]+)\)") + def format_links_for_slack(text: str) -> str: """Convert Markdown links to Slack markdown link format. Args: text (str): The input text that may include Markdown links. Returns: str: Text with Markdown links converted to Slack markdown links. """ if not text: return text - markdown_link_pattern = re.compile(r"\[([^\]]+)\]\((https?://[^\s)]+)\)") - return markdown_link_pattern.sub(r"<\2|\1>", text) + return _MARKDOWN_LINK_PATTERN.sub(r"<\2|\1>", text)
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (10)
backend/apps/ai/agent/tools/rag/generator.py(2 hunks)backend/apps/ai/common/constants.py(1 hunks)backend/apps/core/models/prompt.py(1 hunks)backend/apps/slack/blocks.py(2 hunks)backend/apps/slack/common/question_detector.py(2 hunks)backend/apps/slack/utils.py(1 hunks)backend/tests/apps/ai/agent/tools/rag/generator_test.py(5 hunks)backend/tests/apps/ai/agent/tools/rag/rag_tool_test.py(2 hunks)backend/tests/apps/ai/management/commands/ai_run_rag_tool_test.py(4 hunks)backend/tests/apps/slack/common/question_detector_test.py(0 hunks)
💤 Files with no reviewable changes (1)
- backend/tests/apps/slack/common/question_detector_test.py
🧰 Additional context used
🧬 Code graph analysis (6)
backend/apps/slack/blocks.py (2)
backend/apps/slack/utils.py (1)
format_links_for_slack(38-52)backend/apps/slack/models/message.py (1)
text(83-85)
backend/apps/slack/common/question_detector.py (1)
backend/apps/core/models/prompt.py (2)
Prompt(14-161)get_slack_question_detector_prompt(154-161)
backend/tests/apps/ai/agent/tools/rag/generator_test.py (1)
backend/apps/ai/agent/tools/rag/generator.py (2)
Generator(14-105)generate_answer(61-105)
backend/apps/ai/agent/tools/rag/generator.py (1)
backend/apps/core/models/prompt.py (2)
Prompt(14-161)get_rag_system_prompt(144-151)
backend/tests/apps/ai/agent/tools/rag/rag_tool_test.py (1)
backend/apps/ai/agent/tools/rag/rag_tool.py (1)
query(34-63)
backend/apps/slack/utils.py (1)
backend/apps/slack/models/message.py (1)
text(83-85)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Run backend tests
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run frontend unit tests
🔇 Additional comments (10)
backend/apps/slack/blocks.py (2)
7-8: LGTM!The import statement is clean and properly structured.
35-35: LGTM! Link formatting integration looks good.The integration of
format_links_for_slackensures that Markdown links are properly converted to Slack's format before being rendered in blocks. This aligns with the PR objective to ensure website links are properly formatted.backend/apps/slack/utils.py (1)
38-52: Handle URLs with parentheses and nested brackets
The patternr"\[([^\]]+)\]\((https?://[^\s)]+)\)"stops at the first)and won’t match URLs containing parentheses or link text with nested brackets (e.g.[Wiki](https://example.com/path(1))or[Example [nested]](https://…)). If you need to support these cases, update the regex (e.g. balanced grouping or a more permissive pattern) and add tests covering them.backend/tests/apps/ai/agent/tools/rag/rag_tool_test.py (1)
125-130: Defaults test coverage looks goodThanks for updating both default-path assertions to the new limit=8 / threshold=0.1 values; this keeps the test suite aligned with the shared constants.
Also applies to: 153-158
backend/apps/ai/common/constants.py (1)
4-5: Updated retrieval defaults make senseBumping the retrieval limit to 8 and allowing similarity scores down to 0.1 matches the test adjustments and should help recall more context for RAG responses.
backend/tests/apps/ai/management/commands/ai_run_rag_tool_test.py (2)
43-43: LGTM! Comments accurately reflect the new defaults.The inline comments correctly document the new constant values (DEFAULT_CHUNKS_RETRIEVAL_LIMIT=8 and DEFAULT_SIMILARITY_THRESHOLD=0.1).
Also applies to: 49-49
109-110: LGTM! Test values correctly updated to match new defaults.All test cases now consistently use the new default values (limit=8, threshold=0.1), ensuring tests remain aligned with the updated constants.
Also applies to: 127-128, 139-141
backend/tests/apps/ai/agent/tools/rag/generator_test.py (3)
105-108: LGTM! Proper mocking of dynamic prompt retrieval.All test methods that invoke
generate_answernow correctly mockPrompt.get_rag_system_prompt, which is necessary since the production code dynamically retrieves the system prompt instead of using a hardcoded constant.Also applies to: 135-138, 161-164, 182-185
202-225: LGTM! Comprehensive test for dynamic prompt integration.The rewritten test effectively verifies:
- The system prompt is retrieved via
Prompt.get_rag_system_prompt()(line 208-210)- The retrieved prompt is correctly passed to OpenAI (lines 223-224)
- The getter is called exactly once (line 225)
This properly validates the refactoring from hardcoded prompts to dynamic retrieval.
230-230: LGTM! Constants test updated for refactored code.The test correctly verifies only
TEMPERATUREnow, asSYSTEM_PROMPThas been removed in favor of dynamic prompt retrieval viaPrompt.get_rag_system_prompt().
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
backend/tests/apps/slack/common/question_detector_test.py (1)
32-34: Consider removing unused parameter.The
monkeypatchparameter is not used in the fixture body. Since the autouse_mock_openaifixture already provides all necessary setup, this parameter is redundant.Apply this diff:
@pytest.fixture - def detector(self, monkeypatch): + def detector(self): """Fixture to provide QuestionDetector instance.""" return QuestionDetector()
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
backend/apps/ai/agent/tools/rag/generator.py(2 hunks)backend/apps/slack/common/question_detector.py(2 hunks)backend/tests/apps/ai/agent/tools/rag/generator_test.py(6 hunks)backend/tests/apps/slack/common/question_detector_test.py(4 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- backend/apps/slack/common/question_detector.py
- backend/apps/ai/agent/tools/rag/generator.py
- backend/tests/apps/ai/agent/tools/rag/generator_test.py
🧰 Additional context used
🧬 Code graph analysis (1)
backend/tests/apps/slack/common/question_detector_test.py (1)
backend/apps/slack/common/question_detector.py (3)
QuestionDetector(18-136)is_owasp_question_with_openai(82-125)is_owasp_question(50-76)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run backend tests
- GitHub Check: Run frontend unit tests
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (4)
backend/tests/apps/slack/common/question_detector_test.py (4)
3-8: LGTM! Necessary imports for enhanced test coverage.The new imports support comprehensive mocking, environment setup, and exception testing for the dynamic prompt retrieval and OpenAI integration.
16-29: LGTM! Solid test isolation strategy.The autouse fixture prevents real OpenAI API calls during basic tests while allowing individual tests to override behavior with specific patches. The prompt mock aligns with the PR's goal to use dynamic prompt retrieval.
229-434: LGTM! Comprehensive test coverage for OpenAI integration.The new tests thoroughly cover:
- Missing/empty prompt validation (ObjectDoesNotExist)
- All response scenarios (YES, NO, empty, None, unexpected)
- API error handling
- Case-insensitive response parsing
The isolated patching strategy in each test ensures proper test independence and aligns with the implementation in
question_detector.py.
436-478: LGTM! Override logic is properly tested.These tests verify the fallback behavior where keyword detection can override OpenAI's NO response, ensuring resilience when the AI model misclassifies OWASP-related questions. The patching strategy appropriately isolates the specific logic being tested.
28932a2
into
OWASP:feature/nestbot-ai-assistant
Proposed change
Resolves #2406
Checklist
make check-testlocally; all checks and tests passed.