Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Bug]: vibe_check_mentor returns generic advice instead of specific TypeScript guidance #185

@kesslerio

Description

@kesslerio

Problem Description

The vibe_check_mentor tool is returning generic, boilerplate advice instead of specific technical guidance when asked about TypeScript 'any' warning reduction strategies. Despite implementing auto-discovery in PR #183, the mentor personas are not analyzing the actual code or providing contextual recommendations.

Current Behavior

When asked about ESLint @typescript-eslint/no-explicit-any warning reduction:

Query: "Should we use automated ESLint --fix tools for reducing TypeScript 'any' type warnings in our production codebase? We have 1027 warnings near the 1030 limit and need to reduce to <500 over 3 months."

Actual Response (Generic):

  • Senior Engineer: "Apply first principles thinking - what's the core problem you're solving?"
  • Product Engineer: "What's the user-facing value? I'd implement just enough to unblock user workflows"
  • AI Engineer: "Modern AI tools can validate our approach before we write code"

Expected Behavior

The mentor should provide specific TypeScript-focused advice:

  • Senior Engineer: "For TypeScript 'any' warnings, avoid auto-fix since any→unknown conversion breaks runtime without type guards. Use ts-migrate for safe markers ($TSFixMe) that preserve functionality."
  • Product Engineer: "1027 warnings near the limit is a production risk. Start with test files where breaking changes are safer. Target 30-40 warnings/week to hit <500 in 3 months."
  • AI Engineer: "Use AI to generate proper type definitions from runtime data. Tools like TypeStat or ts-morph can analyze actual usage patterns."

Root Cause Analysis

The enhanced persona reasoning in vibe_check_mentor_enhanced.py is extracting technical context but not accessing actual TypeScript files to understand:

  1. The specific 'any' usage patterns in the codebase
  2. Whether they're in test files vs production code
  3. The complexity of type conversions needed
  4. Risk levels of different file areas

Evidence

From the debug logs:

[ERROR] MCP server "vibe-check-local" Server stderr: 2025-08-10 21:36:31,952 - __main__ - INFO - No patterns detected for query: Should we use automated ESLint --fix tools...

The system detects NO PATTERNS because it's not reading the actual TypeScript files with 'any' warnings.

Reproduction Steps

  1. Configure vibe-check-local with a TypeScript project
  2. Run: vibe_check_mentor(query="How to reduce TypeScript any warnings?", reasoning_depth="comprehensive")
  3. Observe generic responses instead of TypeScript-specific guidance

Impact

Proposed Solution

  1. Auto-detect TypeScript context: When query mentions TypeScript/ESLint/any, automatically:

    • Find *.ts files in working directory
    • Run grep -l "@typescript-eslint/no-explicit-any" **/*.ts to find affected files
    • Read 2-3 sample files with 'any' usage
  2. Enhanced context extraction: Add TypeScript-specific patterns to ContextExtractor:

    TYPESCRIPT_PATTERNS = {
        'any_warnings': ['@typescript-eslint/no-explicit-any', 'any type', 'type any'],
        'eslint_tools': ['eslint --fix', 'ts-migrate', 'eslint-filtered-fix'],
        'type_safety': ['type guards', 'unknown', 'type assertion', 'as any']
    }
  3. File-aware persona responses: Make personas read actual code:

    # In generate_senior_engineer_response
    if 'typescript' in tech_context.technologies and 'any' in query.lower():
        # Read actual TypeScript files with any usage
        any_files = find_files_with_pattern('any', '*.ts')
        if any_files:
            # Analyze complexity of any usage
            return typescript_specific_advice(any_files)

Priority

P1 - High Priority

This is a regression of the core value proposition - providing specific, contextual advice based on actual code analysis.

Labels

  • bug
  • P1
  • area:mentor
  • area:context-awareness

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions