Thanks to visit codestin.com
Credit goes to github.com

Skip to content
/ django Public

Comments

Fixed #32213 -- Ensured KeyTransform values are properly quoted on SQLite#20487

Open
VIZZARD-X wants to merge 1 commit intodjango:mainfrom
VIZZARD-X:ticket_32213
Open

Fixed #32213 -- Ensured KeyTransform values are properly quoted on SQLite#20487
VIZZARD-X wants to merge 1 commit intodjango:mainfrom
VIZZARD-X:ticket_32213

Conversation

@VIZZARD-X
Copy link
Contributor

@VIZZARD-X VIZZARD-X commented Jan 3, 2026 •
edited
Loading

Trac ticket number

ticket-32213

Branch description

On SQLite, KeyTransform lookups (e.g., .values('data__key')) extract values as raw strings without quotes. This causes json.loads to misinterpret them (e.g., string "123" becomes integer 123, and string "value" is returned as value without quotes).

The solution implements KeyTransform.select_format to apply a CASE WHEN JSON_TYPE guard at the SQL level, which explicitly applies JSON_QUOTE to text values while allowing primitives to fall through naturally. This ensures strict type preservation for edge cases like "null" vs null without introducing coupling in JSONField.from_db_value.

Test:
A regression test test_json_key_transform_type_preservation_sqlite was added to the existing test_jsonfield.py which checks that string, integer, boolean, and null types remain distinct after extraction.

Checklist

  • This PR targets the main branch.
  • The commit message is written in past tense, mentions the ticket number, and ends with a period.
  • I have checked the "Has patch" ticket flag in the Trac system.
  • I have added or updated relevant tests.
  • I have added or updated relevant docs, including release notes if applicable.
  • I have attached screenshots in both light and dark modes for any UI changes.

@VIZZARD-X VIZZARD-X force-pushed the ticket_32213 branch 2 times, most recently from 6171394 to 51be479 Compare January 3, 2026 11:32
charettes
charettes reviewed Jan 3, 2026
View reviewed changes
Copy link
Member

@charettes charettes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we considered implementing KeyTransform.select_format to behave differently on SQLite instead to avoid bi-directionally coupling JSONField with it?

select_format is only called when the expression is part of the SELECT clause so it could be used to generate proper SQL (e.g. adapt quoting) so JSONField.from_db_value can remain unchanged.

This would be a better solution in my opinion as isolates the SQLite specialization to KeyTransform.

tests/model_fields/test_ticket_32213.py Outdated
Comment on lines 5 to 6
class Ticket32213Model(models.Model):
data = models.JSONField()
Copy link
Member

@charettes charettes Jan 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid creating models / tables and ticket classes for each tickets; reuse the existing ones.

django/db/models/fields/json.py Outdated
Comment on lines 95 to 98
if value not in ("true", "false", "null") and not value.startswith(
("{", "[")
):
return value
Copy link
Member

@charettes charettes Jan 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about top level integers and strings?

@VIZZARD-X
Copy link
Contributor Author

VIZZARD-X commented Jan 5, 2026

I've updated the PR based on your feedback:

  • Implemented KeyTransform.select_format: I moved the SQLite-specific logic here to avoid coupling JSONField.from_db_value with backend quirks.
  • The select_format method now uses a CASE WHEN statement to check if the value starts with { or [. If not (e.g., top-level strings or numbers), it applies JSON_QUOTE, ensuring json.loads receives valid JSON.
  • I removed the custom test model and updated the regression test to reuse the existing NullableJSONModel.

Ready for another look 👍

@VIZZARD-X VIZZARD-X marked this pull request as draft January 5, 2026 16:45
@VIZZARD-X VIZZARD-X marked this pull request as ready for review January 5, 2026 17:01
felixxm
felixxm reviewed Jan 5, 2026
View reviewed changes
django/db/models/fields/json.py
f"ELSE JSON_EXTRACT({lhs}, %s) END",
args * 3,
)
return super().select_format(compiler, sql, params)
Copy link
Member

@felixxm felixxm Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this won't work, see the entire discussion in ticket-33820 (especially this comment).

Copy link
Contributor Author

@VIZZARD-X VIZZARD-X Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed comment:3 on ticket-33820. The key difference here is that this implementation explicitly guards WHEN 'text' first.

This forces JSON_QUOTE on strings (preserving "null" as '"null"'), while letting primitives fall through to the ELSE block as raw values. The updated test confirms this successfully disambiguates conflicting pairs like "null" vs null and "123" vs 123.

@VIZZARD-X
Copy link
Contributor Author

VIZZARD-X commented Jan 16, 2026

@charettes I've updated the implementation to use KeyTransform.select_format as suggested. This isolates the SQLite-specific logic from the main JSONField code.

@felixxm regarding the type preservation concerns: I've added a regression test to tests/model_fields/test_jsonfield.py that explicitly verifies that strings (e.g., "123") remain strings and do not collide with integers (123) or booleans on SQLite.

Ready for a review when you are free...

@github-actions
Copy link

github-actions bot commented Jan 16, 2026

📊 Coverage Report for Changed Files

-------------
Diff Coverage
Diff: origin/main...HEAD, staged and unstaged changes
-------------
django/db/models/fields/json.py (88.9%): Missing lines 555
-------------
Total:   9 lines
Missing: 1 line
Coverage: 88%
-------------

Note: Missing lines are warnings only. Some lines may not be covered by SQLite tests as they are database-specific.

For more information about code coverage on pull requests, see the contributing documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@felixxm felixxm felixxm left review comments

+1 more reviewer

@charettes charettes charettes left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@VIZZARD-X @charettes @felixxm