Add regex patterns to JSON schema for `Decimal` type #11987

Dima-Bulavenko · 2025-06-16T04:54:59Z

Change Summary

This PR adds a regular expression pattern to the JSON schema of Decimal fields to validate their string representation according to the max_digits and decimal_places parameters of the Field class.

Updated existing tests for Decimal fields to align with the new validation pattern.
Added new tests to validate the pattern across different combinations of max_digits and decimal_places.

You can run the new tests with the following command:

uv run pytest tests/test_json_schema.py::TestDecimalPattern

Related issue number

fixes #11016
fixes #11420
fixes #10867

Checklist

The pull request title is a good summary of the changes - it will be used in the changelog
Unit tests for the changes exist
Tests pass on CI
Documentation reflects the changes where applicable
My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Selected Reviewer: @Viicos

Old tests have been updated to align with the new validation pattern for Decimal fields in the JSON schema.

…ith new test cases.

…ontain validaton pattern.

hyperlint-ai · 2025-06-16T04:55:25Z

PR Change Summary

Enhanced JSON schema validation for Decimal fields by introducing a regex pattern based on max_digits and decimal_places parameters.

Added regex pattern to JSON schema for validating Decimal fields.
Updated existing tests to reflect the new validation pattern.
Introduced new tests for various combinations of max_digits and decimal_places.

Modified Files

docs/concepts/json_schema.md

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

codspeed-hq · 2025-06-16T05:03:34Z

CodSpeed Performance Report

Merging #11987 will not alter performance

_{Comparing Dima-Bulavenko:decima_constraint (f9ec7a1) with main (9452e13)}

Summary

✅ 46 untouched benchmarks

github-actions · 2025-06-16T05:43:40Z

Coverage report

This PR does not seem to contain any modification to coverable code.

Dima-Bulavenko · 2025-06-16T12:06:05Z

please review - thanks!

Viicos

Thanks for the contribution. I've checked locally and the regex seems to correctly match.

Similar to #11420, we'd have to also add patterns when max_digits xor decimal_places is None. Would you be interested in pursuing?

tests/test_json_schema.py

pydantic/json_schema.py

refactor: Update tests to match the new regex

Dima-Bulavenko · 2025-07-04T04:30:28Z

Thanks for the contribution. I've checked locally and the regex seems to correctly match.

Similar to #11420, we'd have to also add patterns when max_digits xor decimal_places is None. Would you be interested in pursuing?

Thanks for reviewing!

It seems the pattern already handles cases where max_digits or decimal_places is None.

I've added tests to cover these scenarios:

@pytest.fixture
def get_decimal_pattern():
    def pattern(max_digits=None, decimal_places=None) -> str:
        filed = TypeAdapter(Annotated[Decimal, Field(max_digits=max_digits, decimal_places=decimal_places)])
        return filed.json_schema()['anyOf'][1]['pattern']

    return pattern

@pytest.mark.parametrize("valid_decimal", ["0.1", "0000.1", "11.1", "0011.1", "11111111.1"])
def test_decimal_pattern_max_digits_set_none(valid_decimal, get_decimal_pattern):
    max_digits = None
    decimal_places = 1
    pattern = get_decimal_pattern(max_digits=max_digits, decimal_places=decimal_places)
    
    assert re.fullmatch(pattern, valid_decimal) is not None


@pytest.mark.parametrize("valid_decimal", ["0.1", "000.1", "0.001000", "0000.001000"])
def test_decimal_pattern_decimal_places_set_none(valid_decimal, get_decimal_pattern):
    max_digits = 3
    decimal_places = None
    pattern = get_decimal_pattern(max_digits=max_digits, decimal_places=decimal_places)
    
    assert re.fullmatch(pattern, valid_decimal) is not None

@pytest.mark.parametrize("unvalid_decimal", ["0.0001", "111.1", "11.01"])
def test_decimal_pattern_with_unvalid_decimals_decimal_places_set_none(unvalid_decimal, get_decimal_pattern):
    max_digits = 3
    decimal_places = None
    pattern = get_decimal_pattern(max_digits=max_digits, decimal_places=decimal_places)
    
    assert re.fullmatch(pattern, unvalid_decimal) is None

@pytest.mark.parametrize("valid_decimal", ["11111111", "1111.11111", "0.00000001"])
def test_decimal_pattern_decimal_places_max_digits_set_none(valid_decimal, get_decimal_pattern):
    max_digits = None
    decimal_places = None
    pattern = get_decimal_pattern(max_digits=max_digits, decimal_places=decimal_places)
    
    assert re.fullmatch(pattern, valid_decimal) is not None

Viicos · 2025-07-04T07:59:50Z

It seems the pattern already handles cases where max_digits or decimal_places is None.

Sorry what I said was confusing, I wanted to say that it would be nice to have simpler patterns generated for cases where max_digits xor decimal_places is None.

refactor: Delete redundunt tests. test: Create new tests

…s are both set The pattern incorrectly counted trailing zeros in integer strings towards max_digits. For example, "1200" with max_digits=4 and decimal_places=2 would incorrectly match even thought pattern must allow only two integer places.

Dima-Bulavenko · 2025-07-05T02:16:09Z

It seems the pattern already handles cases where max_digits or decimal_places is None.

Sorry what I said was confusing, I wanted to say that it would be nice to have simpler patterns generated for cases where max_digits xor decimal_places is None.

Thank you for clarifying your request.

I've updated the implementation to generate simpler patterns for each combination of max_digits and decimal_places.

Viicos

Given the complexity to implement this correctly (see my comments above), I'm starting to think if we should just define a simpler regex, describing the allowed characters (something like ^(?:\+|-)?[\d.]+$ in validation mode, and ^-?[\d.]+$ in serialization mode). This regex will also allow invalid inputs (but at least we are certain valid inputs wont be disallowed).

Viicos · 2025-07-05T09:17:10Z

pydantic/json_schema.py

+
+            # Case 4: Both are None (no restrictions)
+            else:
+                pattern += r'\d*\.?\d*$'  # look for arbitrary integer or decimal


This allows '.' as an input, which can't be validated.

Thank you for pointing this out.
However, r'\d*\.?\d*$' is not the full pattern used for validation.

The complete pattern includes an additional regex component that prevents '.' from being a valid input:

pydantic/pydantic/json_schema.py

Lines 682 to 684 in 88adb2d

pattern = (

r'^(?!^[-+.]*$)[+-]?0*' # check it is not empty string and not one or sequence of ".+-" characters.

)

Additionally, the edge case for '.' is covered by the following test:

pydantic/tests/test_json_schema.py

Lines 7005 to 7009 in 88adb2d

@pytest.mark.parametrize('invalid_decimal', ['.', '-.', '..', '1.1.1', '0.0.0', '1..1', '-', '--'])

def test_decimal_pattern_reject_invalid_with_decimal_places_max_digits_unset(invalid_decimal, get_decimal_pattern):

pattern = get_decimal_pattern()

assert re.fullmatch(pattern, invalid_decimal) is None

Viicos · 2025-07-05T09:26:46Z

pydantic/json_schema.py

+                )
+
+            # Case 2: Only max_digits is set
+            elif max_digits is not None and decimal_places is None:


With max_digits set to e.g. 5, it wrongfully matches '1000.1111111', etc.

Thank you for your feedback.

From my testing, the pattern appears to work correctly when max_digits=5 and the invalid value '1000.1111111' is rejected as expected.

Here is a test demonstrating this behavior:

@pytest.fixture def get_decimal_pattern(): def pattern(max_digits=None, decimal_places=None) -> str: field = TypeAdapter(Annotated[Decimal, Field(max_digits=max_digits, decimal_places=decimal_places)]) return field.json_schema()['anyOf'][1]['pattern'] return pattern @pytest.mark.parametrize('invalid_decimal', ['1000.1111111']) def test_only_max_digits_set(invalid_decimal, get_decimal_pattern): pattern = get_decimal_pattern(max_digits=5, decimal_places=None) assert re.fullmatch(pattern, invalid_decimal) is None

Let me know if there's a specific case I may have missed!

Dima-Bulavenko · 2025-07-05T17:24:24Z

I think in validation mode, it's better to use the pattern ^[-+]?(?:\d+|\d+\.\d+)$, which allows both integers and decimals of any size. However, if we use this simple pattern, we completely lose the main purpose of this issue (#10867), which is to check for max_digits and decimal_places.

Also, the more complex regex patterns are only created in two cases:

When max_digits is set, or
When both max_digits and decimal_places are not None.

I believe these cases don't come up very often, but if a user sets these options, they probably want the regex pattern to actually check max_digits and decimal_places.

For serialization mode, I agree that a simple regex like ^-?(?:\d+|\d+\.\d+)$ is useful because it shows that the data can be either an integer or a decimal.

Please let me know what you think about this. I should also mention that it's possible I don't fully understand the purpose of the validation and serialization schemas or all the cases where they might be used.

Viicos

Thanks @Dima-Bulavenko, @Jack70248 I'm adding you as a co-author for the initial work.

`tests/test_multi_body_errors.py::test_openapi_schema` failed with pydantic 2.12.0a1 due to pydantic/pydantic#11987. Since we're testing the exact contents of the JSON schema, the easiest fix seems to be to add version-dependent handling for this.

Dima-Bulavenko added 5 commits June 16, 2025 02:37

feat: Add regular expression constraint to decimal JSON schema

fde9d94

test: update existing tests for Decimal JSON schema pattern validation

07db820

Old tests have been updated to align with the new validation pattern for Decimal fields in the JSON schema.

test: Add ADDTESTS comment to mark old test that have to be updated w…

b6d3c4b

…ith new test cases.

test: Tailor print in docs for Model.model_json_schema() so that it c…

aa17cce

…ontain validaton pattern.

test: Create tests to check the validation pattern for decimal string.

b9dc9cf

github-actions bot added the relnotes-fix Used for bugfixes. label Jun 16, 2025

fix: Remove type annotations to pass test on Python version 3.9

e0e7e56

pydantic-hooky bot added the ready for review label Jun 16, 2025

pydantic-hooky bot assigned Viicos Jun 16, 2025

Viicos reviewed Jul 2, 2025

View reviewed changes

tests/test_json_schema.py Outdated Show resolved Hide resolved

Viicos reviewed Jul 2, 2025

View reviewed changes

pydantic/json_schema.py Outdated Show resolved Hide resolved

Viicos added the awaiting author revision awaiting changes from the PR author label Jul 2, 2025

refactor: Delete unwanted escaping "\" character from regex.

5874292

refactor: Update tests to match the new regex

Dima-Bulavenko force-pushed the decima_constraint branch from ca8c4f6 to 5874292 Compare July 4, 2025 03:17

refactor: Flatten tests

73afd01

Dima-Bulavenko requested a review from Viicos July 4, 2025 04:57

Dima-Bulavenko added 3 commits July 5, 2025 00:04

refactor: Reduce size of regex.

7b25a74

refactor: Tailor old test to match reduced patterns

23210d7

refactor: Delete redundunt tests. test: Create new tests

Viicos reviewed Jul 5, 2025

View reviewed changes

Dima-Bulavenko requested a review from Viicos July 8, 2025 14:56

Update test_json_schema.py

f9ec7a1

Viicos changed the title ~~Add regex pattern to JSON schema for validating Decimal fields based on max_digits and decimal_places~~ Add regex patterns to JSON schema for Decimal type Jul 11, 2025

Viicos enabled auto-merge (squash) July 11, 2025 11:48

Viicos approved these changes Jul 11, 2025

View reviewed changes

Viicos added relnotes-feature and removed relnotes-fix Used for bugfixes. labels Jul 11, 2025

Viicos merged commit 4057cd2 into pydantic:main Jul 11, 2025
60 checks passed

cjwatson mentioned this pull request Sep 2, 2025

👽️ Ensure compatibility with Pydantic 2.12.0 fastapi/fastapi#14036

Open

	pattern = (
	r'^(?!^[-+.]$)[+-]?0' # check it is not empty string and not one or sequence of ".+-" characters.
	)

	@pytest.mark.parametrize('invalid_decimal', ['.', '-.', '..', '1.1.1', '0.0.0', '1..1', '-', '--'])
	def test_decimal_pattern_reject_invalid_with_decimal_places_max_digits_unset(invalid_decimal, get_decimal_pattern):
	pattern = get_decimal_pattern()

	assert re.fullmatch(pattern, invalid_decimal) is None

Uh oh!

Add regex patterns to JSON schema for Decimal type #11987

Add regex patterns to JSON schema for Decimal type #11987

Conversation

Dima-Bulavenko commented Jun 16, 2025 • edited by Viicos Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change Summary

Related issue number

Checklist

Uh oh!

hyperlint-ai bot commented Jun 16, 2025

PR Change Summary

Uh oh!

codspeed-hq bot commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #11987 will not alter performance

Summary

Uh oh!

github-actions bot commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage report

Uh oh!

Dima-Bulavenko commented Jun 16, 2025

Uh oh!

Viicos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Dima-Bulavenko commented Jul 4, 2025

Uh oh!

Viicos commented Jul 4, 2025

Uh oh!

Dima-Bulavenko commented Jul 5, 2025

Uh oh!

Viicos left a comment

Choose a reason for hiding this comment

Uh oh!

Viicos Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

Dima-Bulavenko Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

Viicos Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

Dima-Bulavenko Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

Dima-Bulavenko commented Jul 5, 2025

Uh oh!

Viicos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Add regex patterns to JSON schema for `Decimal` type #11987

Add regex patterns to JSON schema for `Decimal` type #11987

Dima-Bulavenko commented Jun 16, 2025 •

edited by Viicos

Loading

codspeed-hq bot commented Jun 16, 2025 •

edited

Loading

github-actions bot commented Jun 16, 2025 •

edited

Loading