Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@scosman
Copy link
Collaborator

@scosman scosman commented Jul 25, 2025

The folks behind uv and ruff have a type checker! It's about 3 billion times faster than pyright so let's use it.

Summary by CodeRabbit

  • Chores
    • Replaced the Python type checker from Pyright to Ty across development workflows, documentation, and configuration files.
    • Updated dependency files to remove Pyright and add Ty as the new type checking tool.
    • Adjusted CI scripts and local check scripts to use Ty for type checking.
    • Improved error handling for enum exhaustiveness by adopting standard Python utilities.
    • Updated documentation, badges, and recommended IDE extensions to reflect the switch to Ty.
    • Disabled the Python language server and related analysis settings in VSCode configuration.
  • Bug Fixes
    • Improved type-checking reliability and error messages in several internal modules.
    • Enhanced error handling and validation for provider and enum-related API endpoints.
  • Refactor
    • Consolidated and simplified error handling for enums using standard assertions.
    • Streamlined type conversion logic for input/output handling in adapters.
  • Tests
    • Updated tests to match new error types and messages resulting from the switch to Ty and assertion-based error handling.
  • Style
    • Fixed formatting and alignment issues in documentation tables.

@scosman scosman requested a review from leonardmq July 25, 2025 18:15
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jul 25, 2025

Walkthrough

This update removes the use of the Pyright type checker in favor of the "ty" type checker throughout the codebase, including CI scripts, documentation, and dependency files. It also replaces a custom function for exhaustive enum checks with the standard assert_never from typing_extensions across core libraries, adjusts related tests, and updates configuration and formatting accordingly.

Changes

Files/Paths Change Summary
.github/workflows/build_and_test.yml, checks.sh, AGENTS.md, CONTRIBUTING.md, README.md Switched Python type checking from Pyright to "ty" in CI, scripts, and documentation; updated table formatting.
pyproject.toml, libs/core/pyproject.toml, libs/server/pyproject.toml Removed Pyright dev dependency and config; added "ty" dependency and config.
libs/core/kiln_ai/utils/exhaustive_error.py Deleted custom raise_exhaustive_enum_error utility function.
libs/core/kiln_ai/adapters/adapter_registry.py, libs/core/kiln_ai/adapters/chat/chat_formatter.py, libs/core/kiln_ai/adapters/eval/base_eval.py, libs/core/kiln_ai/adapters/eval/registry.py, libs/core/kiln_ai/adapters/fine_tune/dataset_formatter.py, libs/core/kiln_ai/adapters/model_adapters/litellm_adapter.py, libs/core/kiln_ai/adapters/parsers/parser_registry.py, libs/core/kiln_ai/adapters/parsers/request_formatters.py, libs/core/kiln_ai/adapters/prompt_builders.py, libs/core/kiln_ai/adapters/provider_tools.py, libs/core/kiln_ai/datamodel/task_output.py, libs/core/kiln_ai/datamodel/eval.py Replaced raise_exhaustive_enum_error with assert_never for exhaustive enum checks; adjusted related logic.
libs/core/kiln_ai/adapters/model_adapters/base_adapter.py, libs/core/kiln_ai/adapters/chat/chat_formatter.py Changed input/output stringification logic: serialize non-strings, return strings as-is.
app/desktop/studio_server/finetune_api.py Added helper for provider validation; centralized error handling for invalid providers.
app/desktop/studio_server/provider_api.py Switched to assert_never for match exhaustiveness; adjusted exception handling and imports.
app/desktop/studio_server/test_finetune_api.py, libs/core/kiln_ai/adapters/parsers/test_parser_registry.py, libs/core/kiln_ai/adapters/parsers/test_request_formatters.py, libs/core/kiln_ai/adapters/model_adapters/test_litellm_adapter.py Updated tests to expect AssertionError or new error messages after switching to assert_never.
libs/core/kiln_ai/datamodel/basemodel.py Added # type: ignore to suppress type checker errors in dynamic method.
.github/workflows/format_and_lint.yml, .vscode/extensions.json, .vscode/settings.json Added formatting check step, updated recommended VSCode extensions, and disabled Python language server.
pyrightconfig.json Removed Pyright configuration file.

Sequence Diagram(s)

sequenceDiagram
    participant Dev as Developer
    participant CI as CI Pipeline
    participant Ty as Ty Type Checker

    Dev->>CI: Push code / open PR
    CI->>Ty: Run "uvx ty check"
    Ty-->>CI: Report type checking results
    CI-->>Dev: Pass/fail status
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

The codebase hops from Pyright to Ty,
With enums now checked by "assert_never"—oh my!
No more custom errors, just typing that's clever,
And docs and scripts all updated together.
The rabbits rejoice in this type-safe terrain,
For cleaner, stricter code is always a gain!
🐇✨

Note

⚡️ Unit Test Generation is now available in beta!

Learn more here, or try it out under "Finishing Touches" below.


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6953f6e and 52f83de.

📒 Files selected for processing (1)
  • .vscode/extensions.json (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • .vscode/extensions.json
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Build Desktop Apps (macos-13)
  • GitHub Check: Build Desktop Apps (ubuntu-22.04-arm)
  • GitHub Check: Build Desktop Apps (macos-latest)
  • GitHub Check: Build Desktop Apps (ubuntu-22.04)
  • GitHub Check: Build Desktop Apps (windows-latest)

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (6)
libs/core/kiln_ai/datamodel/basemodel.py (1)

419-421: Drop the inline annotation instead of silencing the checker

ty flags the free variable child_class because it’s resolved only at runtime.
Rather than adding # type: ignore, omit the return annotation in the function header and rely on the explicit __annotations__ assignment just below, which already provides the correct runtime/type-introspection information.

-        def child_method(self, readonly: bool = False) -> list[child_class]:  # type: ignore
+        # Return annotation is injected after definition to avoid runtime-only symbol issues
+        def child_method(self, readonly: bool = False):
             return child_class.all_children_of_parent_path(self.path, readonly=readonly)

This keeps the signature clean, avoids duplicate annotation writes, and removes the need for a blanket ignore.

app/desktop/studio_server/finetune_api.py (2)

223-230: Consider removing redundant provider validation.

The manual provider validation check (lines 223-227) is now redundant since base_provider_from_str_id performs the same validation. However, the error messages differ:

  • Manual check: "Fine tune provider '{provider}' not found"
  • Helper function: "Invalid provider '{provider}'. Valid providers are: {valid_providers}"

If the more informative error message from the helper function is preferred, the manual check can be removed.

-        if finetune.provider not in finetune_registry:
-            raise HTTPException(
-                status_code=400,
-                detail=f"Fine tune provider '{finetune.provider}' not found",
-            )
         finetune_adapter = base_provider_from_str_id(finetune.provider)

The usage of the helper function itself is correct and improves code consistency.


369-374: Consider removing redundant provider validation.

Similar to the earlier endpoint, there's redundant validation logic here. The manual check (lines 369-373) and the helper function perform the same validation with different error messages:

  • Manual check: "Fine tune provider '{provider}' not found"
  • Helper function: "Invalid provider '{provider}'. Valid providers are: {valid_providers}"

The helper function provides better UX by listing valid options.

-        if request.provider not in finetune_registry:
-            raise HTTPException(
-                status_code=400,
-                detail=f"Fine tune provider '{request.provider}' not found",
-            )
         finetune_adapter_class = base_provider_from_str_id(request.provider)

The helper function usage is correct and improves consistency.

README.md (1)

20-21: Table header is now blank – give the first column a label for clarity & accessibility

The first header cell was emptied (| |). Screen-readers and MD renderers will now announce this column as “blank”, which hurts accessibility and makes the source harder to skim. A succinct label such as Section or Category keeps the intent clear.

-|         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+| Section |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
libs/core/pyproject.toml (1)

44-45: Duplicate dev-dependency declarations – consider hoisting to workspace root

ty is now declared in both libs/core and libs/server. If another sub-package is added we’ll have a third copy. Hoist common dev deps to the workspace-root pyproject.toml under a shared dev group to avoid version drift.

CONTRIBUTING.md (1)

69-70: Give the exact VS Code extension identifier for Ty

Beginners will search unsuccessfully for “Ty”. Add the marketplace ID so the extension can be installed with one click, e.g.:

- Ty (astral type checker)
+ Ty (astral type checker) – `astral-sh.ty-vscode`
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1a6138c and 9572523.

⛔ Files ignored due to path filters (3)
  • libs/core/uv.lock is excluded by !**/*.lock
  • libs/server/uv.lock is excluded by !**/*.lock
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (29)
  • .github/workflows/build_and_test.yml (1 hunks)
  • AGENTS.md (1 hunks)
  • CONTRIBUTING.md (1 hunks)
  • README.md (1 hunks)
  • app/desktop/studio_server/finetune_api.py (5 hunks)
  • app/desktop/studio_server/provider_api.py (5 hunks)
  • app/desktop/studio_server/test_finetune_api.py (1 hunks)
  • checks.sh (1 hunks)
  • libs/core/kiln_ai/adapters/adapter_registry.py (2 hunks)
  • libs/core/kiln_ai/adapters/chat/chat_formatter.py (3 hunks)
  • libs/core/kiln_ai/adapters/eval/base_eval.py (2 hunks)
  • libs/core/kiln_ai/adapters/eval/registry.py (2 hunks)
  • libs/core/kiln_ai/adapters/fine_tune/dataset_formatter.py (2 hunks)
  • libs/core/kiln_ai/adapters/model_adapters/base_adapter.py (1 hunks)
  • libs/core/kiln_ai/adapters/model_adapters/litellm_adapter.py (5 hunks)
  • libs/core/kiln_ai/adapters/model_adapters/test_litellm_adapter.py (1 hunks)
  • libs/core/kiln_ai/adapters/parsers/parser_registry.py (2 hunks)
  • libs/core/kiln_ai/adapters/parsers/request_formatters.py (2 hunks)
  • libs/core/kiln_ai/adapters/parsers/test_parser_registry.py (1 hunks)
  • libs/core/kiln_ai/adapters/parsers/test_request_formatters.py (1 hunks)
  • libs/core/kiln_ai/adapters/prompt_builders.py (2 hunks)
  • libs/core/kiln_ai/adapters/provider_tools.py (1 hunks)
  • libs/core/kiln_ai/datamodel/basemodel.py (1 hunks)
  • libs/core/kiln_ai/datamodel/eval.py (3 hunks)
  • libs/core/kiln_ai/datamodel/task_output.py (2 hunks)
  • libs/core/kiln_ai/utils/exhaustive_error.py (0 hunks)
  • libs/core/pyproject.toml (1 hunks)
  • libs/server/pyproject.toml (1 hunks)
  • pyproject.toml (2 hunks)
💤 Files with no reviewable changes (1)
  • libs/core/kiln_ai/utils/exhaustive_error.py
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py

📄 CodeRabbit Inference Engine (.cursor/rules/project.mdc)

**/*.py: Always assume pydantic 2 (not pydantic 1)
The project supports Python 3.10 and above

Files:

  • libs/core/kiln_ai/adapters/parsers/test_request_formatters.py
  • libs/core/kiln_ai/datamodel/task_output.py
  • libs/core/kiln_ai/adapters/parsers/request_formatters.py
  • libs/core/kiln_ai/adapters/adapter_registry.py
  • libs/core/kiln_ai/adapters/eval/registry.py
  • libs/core/kiln_ai/adapters/eval/base_eval.py
  • app/desktop/studio_server/test_finetune_api.py
  • libs/core/kiln_ai/adapters/parsers/parser_registry.py
  • libs/core/kiln_ai/adapters/parsers/test_parser_registry.py
  • libs/core/kiln_ai/adapters/provider_tools.py
  • libs/core/kiln_ai/adapters/model_adapters/test_litellm_adapter.py
  • libs/core/kiln_ai/datamodel/basemodel.py
  • libs/core/kiln_ai/adapters/prompt_builders.py
  • app/desktop/studio_server/provider_api.py
  • libs/core/kiln_ai/datamodel/eval.py
  • libs/core/kiln_ai/adapters/model_adapters/litellm_adapter.py
  • libs/core/kiln_ai/adapters/fine_tune/dataset_formatter.py
  • libs/core/kiln_ai/adapters/model_adapters/base_adapter.py
  • libs/core/kiln_ai/adapters/chat/chat_formatter.py
  • app/desktop/studio_server/finetune_api.py
**/test_*.py

📄 CodeRabbit Inference Engine (.cursor/rules/project.mdc)

**/test_*.py: Always use pytest for tests in Python code
Test brevity is important. Use approaches for re-use and brevity including using fixtures for repeated code, and using pytest parameterize for similar tests

Files:

  • libs/core/kiln_ai/adapters/parsers/test_request_formatters.py
  • app/desktop/studio_server/test_finetune_api.py
  • libs/core/kiln_ai/adapters/parsers/test_parser_registry.py
  • libs/core/kiln_ai/adapters/model_adapters/test_litellm_adapter.py
🧠 Learnings (7)
AGENTS.md (1)

Learnt from: CR
PR: Kiln-AI/Kiln#0
File: .cursor/rules/project.mdc:0-0
Timestamp: 2025-07-21T15:28:29.450Z
Learning: To run backend / Python tests, run uv run python3 -m pytest . -s -v or a variant of it to run individual tests

.github/workflows/build_and_test.yml (1)

Learnt from: CR
PR: Kiln-AI/Kiln#0
File: .cursor/rules/project.mdc:0-0
Timestamp: 2025-07-21T15:28:29.450Z
Learning: To run backend / Python tests, run uv run python3 -m pytest . -s -v or a variant of it to run individual tests

libs/core/kiln_ai/datamodel/task_output.py (3)

Learnt from: leonardmq
PR: #418
File: libs/core/kiln_ai/adapters/ml_model_list.py:0-0
Timestamp: 2025-07-16T09:37:39.816Z
Learning: The glm_z1_rumination_32b_0414 model was intentionally removed from the built_in_models list due to output formatting issues: output was duplicated in both output and reasoning fields, and contained random internal JSON in the output. This model should not be re-added without addressing these formatting problems.

Learnt from: leonardmq
PR: #418
File: libs/core/kiln_ai/adapters/ml_model_list.py:0-0
Timestamp: 2025-07-16T09:37:39.816Z
Learning: The glm_z1_rumination_32b_0414 model was intentionally removed from the built_in_models list due to output formatting issues: output was duplicated in both output and reasoning fields, and contained random internal JSON in the output. This model should not be re-added without addressing these formatting problems.

Learnt from: leonardmq
PR: #313
File: libs/core/kiln_ai/adapters/extraction/gemini_extractor.py:86-88
Timestamp: 2025-05-15T17:16:16.742Z
Learning: In Kiln's extraction configuration, Pydantic automatically coerces string keys (e.g., "document") into Kind enum values when validating dictionary fields typed as dict[Kind, str], so no manual conversion is needed when accessing these values.

CONTRIBUTING.md (1)

Learnt from: CR
PR: Kiln-AI/Kiln#0
File: .cursor/rules/project.mdc:0-0
Timestamp: 2025-07-21T15:28:29.450Z
Learning: Applies to **/*.{js,jsx} : We always use TypeScript, not JavaScript

checks.sh (1)

Learnt from: CR
PR: Kiln-AI/Kiln#0
File: .cursor/rules/project.mdc:0-0
Timestamp: 2025-07-21T15:28:29.450Z
Learning: To run backend / Python tests, run uv run python3 -m pytest . -s -v or a variant of it to run individual tests

libs/core/kiln_ai/adapters/provider_tools.py (1)

Learnt from: leonardmq
PR: #315
File: libs/core/kiln_ai/adapters/extractors/registry.py:3-3
Timestamp: 2025-06-01T17:11:57.451Z
Learning: The from google import genai import is valid and working correctly in the Kiln AI codebase, despite static analysis tools potentially flagging it as an error.

libs/core/kiln_ai/datamodel/eval.py (1)

Learnt from: leonardmq
PR: #313
File: libs/core/kiln_ai/adapters/extraction/gemini_extractor.py:86-88
Timestamp: 2025-05-15T17:16:16.742Z
Learning: In Kiln's extraction configuration, Pydantic automatically coerces string keys (e.g., "document") into Kind enum values when validating dictionary fields typed as dict[Kind, str], so no manual conversion is needed when accessing these values.

🧬 Code Graph Analysis (2)
libs/core/kiln_ai/adapters/parsers/test_parser_registry.py (1)
libs/core/kiln_ai/adapters/parsers/parser_registry.py (1)
  • model_parser_from_id (8-20)
app/desktop/studio_server/finetune_api.py (2)
libs/core/kiln_ai/adapters/fine_tune/base_finetune.py (3)
  • BaseFinetuneAdapter (40-182)
  • FineTuneParameter (21-29)
  • FineTuneStatus (12-18)
app/web_ui/src/lib/types.ts (1)
  • FineTuneParameter (19-19)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Build Desktop Apps (ubuntu-22.04-arm)
  • GitHub Check: Build Desktop Apps (macos-latest)
  • GitHub Check: Build Desktop Apps (ubuntu-22.04)
  • GitHub Check: Build Desktop Apps (windows-latest)
  • GitHub Check: Build Desktop Apps (macos-13)
🔇 Additional comments (38)
app/desktop/studio_server/test_finetune_api.py (1)

369-370: LGTM: Test correctly updated to match new error message format.

The change from exact string match to substring containment check is appropriate and aligns with the centralized provider validation implementation mentioned in the AI summary. The test still validates the core behavior (400 status code for invalid provider) while accommodating the new error message format that includes a list of valid providers.

app/desktop/studio_server/provider_api.py (4)

28-28: LGTM: Proper import of standard assert_never function.

Good practice to use the standard library's assert_never from typing_extensions instead of a custom function for exhaustive enum checking.


48-48: LGTM: Correct exception import.

The change from requests.exceptions.ConnectionError to requests.ConnectionError is the proper way to import and catch this exception type.


311-311: LGTM: Proper use of assert_never for exhaustive enum checking.

The replacement of the custom exhaustive enum error function with the standard assert_never follows Python best practices and provides better type safety.


366-367: LGTM: Consistent use of assert_never with improved comment.

Good replacement of the custom function with standard assert_never, and the comment clearly explains the purpose of the exhaustive check.

app/desktop/studio_server/finetune_api.py (2)

8-12: LGTM! Import addition supports proper type hinting.

The addition of BaseFinetuneAdapter import is necessary for the new helper function's return type annotation.


296-297: LGTM! Improved error handling for invalid provider IDs.

This replacement adds proper validation and error handling where there previously would have been a KeyError for invalid provider IDs. The helper function will now return a clear 400 error instead of a generic 500 error.

README.md (1)

25-25: Verify the new Ty badge URL

The badge now points to a personal-gist URL:

https://gist.githubusercontent.com/scosman/.../ty_badge.json

Gists are easy to delete or move, which would break the badge.
Consider hosting the JSON in the repo (e.g. under .badges/ty.json) or on a project-controlled URL to ensure long-term stability.

AGENTS.md (1)

15-15: Confirm CI treats Ty warnings as failures if desired

uvx ty check exits with 0 when it emits warnings but no errors.
Pyright previously allowed --level error to make the pipeline strict.
If you relied on that behaviour, add Ty’s flag:

uvx ty check --strict

(or update the docs to mention that warnings are acceptable).

libs/core/kiln_ai/adapters/parsers/test_parser_registry.py (1)

16-19: LGTM! Test correctly updated for assert_never behavior.

The test properly expects AssertionError with the "Expected code to be unreachable" message, which aligns with the assert_never implementation in the corresponding model_parser_from_id function.

.github/workflows/build_and_test.yml (1)

40-40: LGTM! Correctly updated to use Ty type checker.

The workflow properly switches from uv run pyright . to uvx ty check, aligning with the PR's objective to migrate from Pyright to the Ty type checker.

libs/core/kiln_ai/adapters/parsers/test_request_formatters.py (1)

75-75: LGTM! Test correctly updated for assert_never behavior.

The test properly expects AssertionError with regex match for "Expected code to be unreachable", which is consistent with the assert_never implementation pattern used throughout this refactor.

libs/core/kiln_ai/datamodel/task_output.py (2)

6-6: LGTM! Correctly imports assert_never for exhaustive enum handling.

The import properly adds assert_never from typing_extensions, replacing the custom exhaustive error function with the standard typing utility.


44-44: LGTM! Properly replaced custom error with assert_never.

The default case correctly uses assert_never(rating_type) instead of the custom raise_exhaustive_enum_error, maintaining exhaustive enum checking with the standard typing utility.

libs/core/kiln_ai/adapters/eval/registry.py (2)

1-1: LGTM! Correctly imports assert_never for exhaustive enum handling.

The import properly adds assert_never from typing_extensions to replace the custom exhaustive error function.


17-17: LGTM! Properly replaced custom error with assert_never.

The default case correctly uses assert_never(eval_config_type) instead of the custom error function, maintaining exhaustive enum checking with the standard typing utility.

libs/core/kiln_ai/adapters/parsers/parser_registry.py (1)

1-1: Excellent migration to standard library function

Great change replacing the custom raise_exhaustive_enum_error with assert_never from typing_extensions. This provides better type checking support and is more idiomatic Python code.

Also applies to: 20-20

libs/core/kiln_ai/adapters/adapter_registry.py (1)

3-4: LGTM - Consistent with codebase refactor

Good migration to assert_never for exhaustive enum handling, consistent with the broader codebase changes.

Also applies to: 200-200

checks.sh (1)

16-16: Improved script clarity and ty integration

Good improvements to the script:

  • More descriptive header separating linting/formatting from type checking
  • Proper integration of the ty type checker with uvx ty check
  • Clear separation of concerns in the output

Also applies to: 21-23

libs/core/kiln_ai/adapters/provider_tools.py (1)

387-388: Good migration with helpful comment

Excellent change to assert_never with an updated comment that clearly explains the type checking benefit. This maintains exhaustive enum handling while using standard library functionality.

libs/core/kiln_ai/adapters/prompt_builders.py (1)

3-4: Consistent migration completing the refactor

Perfect completion of the assert_never migration pattern. The comment correctly explains the type checking benefit, and this change maintains consistency across the entire codebase.

Also applies to: 429-429

libs/core/kiln_ai/adapters/fine_tune/dataset_formatter.py (1)

8-8: LGTM: Correct adoption of standard exhaustive enum checking.

The replacement of the custom raise_exhaustive_enum_error function with assert_never from typing_extensions is the correct approach for exhaustive enum handling in Python. This change provides better type checking support and follows standard typing practices.

Also applies to: 97-97

libs/core/kiln_ai/adapters/parsers/request_formatters.py (1)

4-4: LGTM: Consistent adoption of standard exhaustive enum checking.

The change from raise_exhaustive_enum_error to assert_never is correctly implemented and consistent with the broader refactor. This improves type checking support while maintaining the same runtime behavior.

Also applies to: 41-41

libs/core/kiln_ai/adapters/eval/base_eval.py (1)

5-5: LGTM: Proper implementation of standard exhaustive enum checking.

The replacement of raise_exhaustive_enum_error with assert_never is correctly implemented. This change standardizes exhaustive enum handling using the typing utilities while maintaining the same logical behavior.

Also applies to: 162-162

libs/core/kiln_ai/adapters/model_adapters/test_litellm_adapter.py (1)

346-347: LGTM: Test correctly updated to match assert_never behavior.

The test has been properly updated to expect AssertionError with the message "Expected code to be unreachable", which is the correct behavior when assert_never is called. This change aligns with the implementation switch from custom error handling to standard typing utilities.

pyproject.toml (1)

46-47: LGTM: Appropriate ty configuration with sensible exclusions.

The ty configuration correctly excludes test files, build artifacts, web UI components, and virtual environments from type checking. This is a sensible setup that focuses type checking on the core source code.

libs/core/kiln_ai/adapters/model_adapters/base_adapter.py (1)

259-266: LGTM: Improved serialization consistency

The updated logic now consistently preserves string inputs/outputs as-is while serializing all non-string types to JSON. This is more robust than the previous approach that only handled dictionaries, and aligns with similar changes across the codebase for consistent serialization behavior.

libs/core/kiln_ai/adapters/model_adapters/litellm_adapter.py (5)

2-2: LGTM: Correct import updates for type safety improvements

The addition of cast and assert_never imports aligns with the broader refactor to standardize exhaustive enum handling and resolve type inference issues.

Also applies to: 7-7


155-162: LGTM: Type cast resolves inference issue

The cast is appropriately documented as a workaround for a specific Ty type checker issue. This ensures type safety while the underlying issue is resolved.


195-195: LGTM: Standardized exhaustive enum handling

Replacing the custom error function with assert_never follows the standard typing pattern for exhaustive enum matching and improves type safety.


308-310: LGTM: Type cast improves match statement inference

The cast to ModelProviderName before the match statement helps the type checker understand the exhaustive nature of the enum matching.


344-344: LGTM: Consistent exhaustive enum handling

This completes the standardization of exhaustive enum error handling throughout the file, replacing custom error functions with the standard assert_never pattern.

libs/core/kiln_ai/datamodel/eval.py (3)

3-3: LGTM: Standard import updates for type safety

The addition of cast and assert_never imports supports the type safety improvements and exhaustive enum handling standardization.

Also applies to: 6-6


138-147: LGTM: Improved type safety and defensive programming

The type cast resolves inference issues with the new type checker, and the added guard clause for None parent_eval_config improves runtime safety by preventing potential errors downstream.


196-197: LGTM: Well-documented exhaustive enum handling

The replacement with assert_never follows the standard pattern, and the comment clearly explains that this case should never occur due to the exhaustive enum matching above.

libs/core/kiln_ai/adapters/chat/chat_formatter.py (3)

8-9: LGTM: Standard import for exhaustive enum handling

The addition of assert_never import supports the standardization of exhaustive enum handling across the codebase.


219-219: LGTM: Standardized exhaustive enum handling

Replacing the custom error function with assert_never follows the standard typing pattern for exhaustive enum matching and improves type safety.


231-234: LGTM: Improved serialization logic clarity

The inverted logic is more readable and follows a clear pattern: preserve strings as-is, serialize everything else to JSON. This aligns with similar changes in other modules for consistent serialization behavior.

Comment on lines +37 to 38
"ty>=0.0.1a11",
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Alpha pre-release pin of Ty may cause unexpected breakage

ty>=0.0.1a11 is still an a (alpha) build. Upstream has been shipping breaking API changes every few days. For CI stability you may want to:

  1. Pin to the exact version used in CI (e.g. ==0.0.1a11) and bump deliberately, or
  2. Track the latest pre-release but add a dependabot rule so breakage is caught quickly.
-    "ty>=0.0.1a11",
+    # Pin exact version for reproducible dev installs
+    "ty==0.0.1a11",

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "Fetching release tags for 'ty' from PyPI..."
curl -s https://pypi.org/pypi/ty/json \
  | jq -r '.releases | keys[]' \
  | sort -V \
  | tail -n 10

Length of output: 301


Pin Ty pre-release dependency to avoid unexpected breakage

PyPI now hosts Ty up through 0.0.1a16, so using

"ty>=0.0.1a11"

will automatically pull in 0.0.1a12–a16 (or future alphas) and risks CI instability when upstream makes breaking API changes. For reproducible installs, please choose one of:

  • Pin the exact version used in CI:
    -    "ty>=0.0.1a11",
    +    # Pin exact alpha version for reproducibility
    +    "ty==0.0.1a11",
  • Alternatively, lock to the current alpha series to prevent auto-upgrades beyond a11:
    -    "ty>=0.0.1a11",
    +    # Allow only 0.0.1a11 installs (no newer alphas)
    +    "ty>=0.0.0.1a11,<0.0.1a12",
  • Or continue tracking the latest pre-release but add a Dependabot or Renovate rule to catch breaking changes immediately.
🤖 Prompt for AI Agents
In libs/server/pyproject.toml around lines 37 to 38, the dependency on Ty is
specified as "ty>=0.0.1a11", which allows automatic upgrades to newer
pre-release versions that may introduce breaking changes. To fix this, pin the
Ty dependency to a specific version used in CI (e.g., "ty==0.0.1a11") or
restrict it to the current alpha series (e.g., "ty>=0.0.1a11,<0.0.2") to ensure
reproducible installs and prevent unexpected breakage.

@leonardmq
Copy link
Collaborator

Should we also disable the Python language server in the .vscode/settings.json file?

  "python.languageServer": "None"

They recommend disabling language server in their Cursor / VS Code extension page:

We might also want to add a .vscode/extensions.json file to automatically recommend installing the extensions we want in Cursor / VS Code:

@scosman
Copy link
Collaborator Author

scosman commented Jul 29, 2025

@CodeRabbit review

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jul 29, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
.github/workflows/format_and_lint.yml (2)

37-37: Consider caching the uv sync installation step
uv sync re-resolves and installs all dependencies on every run. Adding a cache keyed on the lock-file hash (e.g. .python-dependencies.lock) can trim several minutes off CI.


47-48: Pin Ty’s version to avoid CI breakage
uvx ty check assumes Ty is present and compatible. To prevent sudden failures when a new Ty release lands, declare an explicit version range in the dev-dependencies section of pyproject.toml, e.g.:

-[tool.poetry.group.dev.dependencies]
-ty = "*"
+ty = "==0.1.*"

(or the equivalent for your build tool).

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9572523 and 2b06ed1.

📒 Files selected for processing (1)
  • .github/workflows/format_and_lint.yml (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
.github/workflows/format_and_lint.yml (1)

Learnt from: CR
PR: Kiln-AI/Kiln#0
File: .cursor/rules/project.mdc:0-0
Timestamp: 2025-07-21T15:28:29.450Z
Learning: To run backend / Python tests, run uv run python3 -m pytest . -s -v or a variant of it to run individual tests

🔇 Additional comments (2)
.github/workflows/format_and_lint.yml (2)

39-42: Lint step only checks import sorting – verify this is deliberate
uvx ruff check --select I . restricts diagnostics to the “I” (isort) rule set. If the intention is full lint coverage, drop --select I and rely on the project’s ruff.toml to drive rule selection.


43-46: Formatting gate with ruff format --check looks good
The new step enforces code formatting in CI without mutating the tree – nice addition.

…ur of ty! Much less memory.

Update contribting.md and .vscode/extensions.json to match
@scosman
Copy link
Collaborator Author

scosman commented Jul 29, 2025

Should we also disable the Python language server in the .vscode/settings.json file?

  "python.languageServer": "None"

They recommend disabling language server in their Cursor / VS Code extension page:

We might also want to add a .vscode/extensions.json file to automatically recommend installing the extensions we want in Cursor / VS Code:

Both sounds good! integrated

@scosman
Copy link
Collaborator Author

scosman commented Jul 29, 2025

@leonardmq made those changes, please review when you have a second.

Thoughts on merging this vs waiting for ty to hit beta?

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
.vscode/extensions.json (1)

1-22: Minor style nit – consistent indentation

This file mixes tabs and spaces; VS Code’s default “JSON with Comments” formatter will normalise to spaces.
Run Format-on-Save once to avoid noisy diffs later.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2b06ed1 and 6953f6e.

📒 Files selected for processing (6)
  • .vscode/extensions.json (1 hunks)
  • .vscode/settings.json (1 hunks)
  • CONTRIBUTING.md (1 hunks)
  • app/desktop/studio_server/finetune_api.py (5 hunks)
  • app/desktop/studio_server/provider_api.py (5 hunks)
  • pyrightconfig.json (0 hunks)
💤 Files with no reviewable changes (1)
  • pyrightconfig.json
🚧 Files skipped from review as they are similar to previous changes (3)
  • CONTRIBUTING.md
  • app/desktop/studio_server/provider_api.py
  • app/desktop/studio_server/finetune_api.py
🧰 Additional context used
🪛 Biome (2.1.2)
.vscode/extensions.json

[error] 15-15: Expected an array, an object, or a literal but instead found ']'.

Expected an array, an object, or a literal here.

(parse)


[error] 21-21: Expected an array, an object, or a literal but instead found ']'.

Expected an array, an object, or a literal here.

(parse)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Build Desktop Apps (ubuntu-22.04-arm)
  • GitHub Check: Build Desktop Apps (ubuntu-22.04)
  • GitHub Check: Build Desktop Apps (macos-13)
  • GitHub Check: Build Desktop Apps (macos-latest)
  • GitHub Check: Build Desktop Apps (windows-latest)
🔇 Additional comments (1)
.vscode/settings.json (1)

18-18: Confirm python.languageServer value is still respected by the latest Python extension

"python.languageServer": "None" was deprecated in recent versions of the MS Python extension; the documented fallback is now "Default".
Please double-check that “None” still disables Pylance in your current toolchain; otherwise developers may silently get Pylance re-enabled.

Copy link
Collaborator

@leonardmq leonardmq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That it does not support typing on self (which seems basic) yet is a little concerning as we might run into similar problems later when we do not expect it.

That said, I tried this new config with a few PRs (including the "allow unicode in model names" one, which has some weird Pydantic typing), and ty seems to work well enough with what we have, so probably fine to start using it now.

In any case, easy to rollback if we ever need to - so maybe let's do it


Edit: actually found a problem where it does not flag missing args into a Pydantic object when the class is defined like this:

class PydanticObjectWithArbitraryObjectFields(BaseModel):
    model_config = ConfigDict(arbitrary_types_allowed=True)

    xxx: SomeObjectType = Field(
        description="The rag config to use for the pipeline",
    )

# does not detect the args are missing in the instantiation (pyright does catch it)
x = PydanticObjectWithArbitraryObjectFields()

@scosman
Copy link
Collaborator Author

scosman commented Jul 29, 2025

@leonardmq yeah lets wait a bit for beta, or at least self 😀. But I'm excited. So fast, so much less memory.

@scosman scosman changed the title Add Ty type checking Add Ty type checking [awaiting Ty beta] Jul 29, 2025
@scosman
Copy link
Collaborator Author

scosman commented Jul 29, 2025

awaiting astral-sh/ty#159 and beta designation.

search and kill typed_self once that's done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants