Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix: regression where _deep_merge() mutates list values in passed dictionary #2400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Jun 3, 2025
Merged

fix: regression where _deep_merge() mutates list values in passed dictionary #2400

merged 4 commits into from
Jun 3, 2025

Conversation

cosmo-grant
Copy link
Contributor

#2318 introduced a regression, by changing from

result[key] = result[key] + value

in _deep_merge() in src/datamodel_code_generator/parser/jsonschema.py to

result[key] += value

because of adopting ruff's unsafe rule PLR6104.

In the old / good version, we create a new list and assign it to result[key]. In the new / bad version, we mutate result[key] in place, which results in incorrect models being generated in some cases.

For example:

{
  "type": "object",
  "properties": {
    "foo": {
      "type": "object",
      "required": [
        "bar"
      ],
      "anyOf": [{
          "required": [
            "baz"
          ]
        },
        {
          "required": [
            "qux"
          ]
        }
      ],
      "properties": {
        "bar": {
          "type": "integer"
        },
        "baz": {
          "type": "integer"
        },
        "qux": {
          "type": "integer"
        }
      }
    }
  }
}

should generate

# generated by datamodel-codegen:
#   filename:  my_schema.json
#   timestamp: 2025-05-09T22:13:24+00:00

from __future__ import annotations

from typing import Optional, Union

from pydantic import BaseModel


class Foo(BaseModel):
    bar: int
    baz: int
    qux: Optional[int] = None


class Foo1(BaseModel):
    bar: int
    baz: Optional[int] = None
    qux: int


class Model(BaseModel):
    foo: Optional[Union[Foo, Foo1]] = None

but currently actually generates

# generated by datamodel-codegen:
#   filename:  my_schema.json
#   timestamp: 2025-05-09T22:13:24+00:00

from __future__ import annotations

from typing import Optional, Union

from pydantic import BaseModel


class Foo(BaseModel):
    bar: int
    baz: int
    qux: Optional[int] = None


class Foo1(BaseModel):
    bar: int
    baz: int
    qux: int


class Model(BaseModel):
    foo: Optional[Union[Foo, Foo1]] = None

This PR reverts to

result[key] = result[key] + value

and adds a unit test.

Fixes #2399.

@cosmo-grant cosmo-grant mentioned this pull request May 9, 2025
Closed
@simontaurus simontaurus mentioned this pull request May 18, 2025
Open
Cosmo Grant added 3 commits June 1, 2025 15:46
@cosmo-grant
no-op change to uv.lock at uv's insistence
85e1fba 
On a fresh clone I got this error:

```
% tox run -e dev
dev: uv-sync> uv sync --locked --python-preference system --extra all --no-default-groups --group dev -p /Users/cosmo.grant/.local/share/uv/tools/tox/bin/python
Resolved 133 packages in 5.04s
error: The lockfile at `uv.lock` needs to be updated, but `--locked` was provided. To update the lockfile, run `uv lock`.
dev: exit 2 (5.11 seconds) /Users/cosmo.grant/git/personal/datamodel-code-generator> uv sync --locked --python-preference system --extra all --no-default-groups --group dev -p /Users/cosmo.grant/.local/share/uv/tools/tox/bin/python pid=46498
  dev: FAIL code 2 (5.13 seconds)
  evaluation failed :( (5.53 seconds)
```

So I ran `uv lock` as the error message suggests.
@cosmo-grant
fix regression where list is mutated unintentionally
446c3af
@cosmo-grant
test jsonschema with required and anyOf required
27265c2
@cosmo-grant cosmo-grant force-pushed the fix_regression_where_list_is_mutated_unintentionally branch from c93aee7 to 27265c2 Compare June 1, 2025 14:46
@codspeed-hq CodSpeed HQ
Copy link

codspeed-hq bot commented Jun 3, 2025 •
edited
Loading

CodSpeed Performance Report

Merging #2400 will not alter performance

Comparing cosmo-grant:fix_regression_where_list_is_mutated_unintentionally (8668e22) with main (f456147)

Summary

✅ 31 untouched benchmarks
🆕 1 new benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
🆕 test_main_jsonschema_required_and_any_of_required N/A 60.5 ms N/A

@codecov Codecov
Copy link

codecov bot commented Jun 3, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.10%. Comparing base (f456147) to head (8668e22).
Report is 1 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #2400   +/-   ##
=======================================
  Coverage   98.10%   98.10%           
=======================================
  Files          65       65           
  Lines        8032     8040    +8     
  Branches      810      810           
=======================================
+ Hits         7880     7888    +8     
  Misses        115      115           
  Partials       37       37
Flag Coverage Δ
unittests 98.10% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@koxudaxi koxudaxi merged commit be0f22f into koxudaxi:main Jun 3, 2025
32 checks passed
@koxudaxi
Copy link
Owner

koxudaxi commented Jun 3, 2025

I'm sorry for my late reply.
LGTM.
Thank you very much!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Regression where _deep_merge() mutates list values in passed dictionary
2 participants
@cosmo-grant @koxudaxi