Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Form] Fix precision of MoneyToLocalizedStringTransformer's divisions and multiplications #24036

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 29, 2017
Merged

[Form] Fix precision of MoneyToLocalizedStringTransformer's divisions and multiplications #24036

merged 1 commit into from
Sep 29, 2017

Conversation

Rubinum
Copy link
Contributor

@Rubinum Rubinum commented Aug 30, 2017

Q A
Branch? master
Bug fix? yes
New feature? no
BC breaks? no
Deprecations? no
Tests pass? no
License MIT

There is a PHP Bug with the accuracy of divisions and multiplications when /= and *= are used.
Here is the proof: https://3v4l.org/u1DkX
It would be better to use bcmul() and bcdiv() in the MoneyToLocalizedStringTransformer.php to prevent this bug.

stof
stof previously requested changes Aug 30, 2017
@@ -11,6 +11,7 @@

namespace Symfony\Component\Form\Extension\Core\DataTransformer;

use function function_exists;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be removed

$transformer = new MoneyToLocalizedStringTransformer(null, null, null, 100);

if (!function_exists('bcmul')) {
$this->markTestSkipped('Test will only work when bcmath-extension is enabled');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use @requires function bcmul (or @requires extension bcmath) instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be removed now that the code doesn't use bcmath functions anymore

@xabbuh xabbuh added this to the 2.7 milestone Aug 30, 2017
@xabbuh
Copy link
Member

xabbuh commented Aug 30, 2017

IMO this is a bugfix and should then be based on the 2.7 branch.

@stof
Copy link
Member

stof commented Aug 30, 2017

and even if we decide to consider it as a feature, it should go in 3.4, not in master (which is currently 4.0)

if (!function_exists('bcmul')) {
$this->markTestSkipped('Test will only work when bcmath-extension is enabled');
}
$this->assertEquals(3655, $transformer->reverseTransform('36.55'));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I don't need this assertions because $this->assertSame(3655, (int) $transformer->reverseTransform('36.55')); should fullfill this case. I will remove it if you don't have objections.

@nicolas-grekas
Copy link
Member

We should try harder to fix the issue for everyone without resorting to bcmath: the function_exists() check means some ppl will have the correct behavior, and some others will have the bug. This wouldn't qualify as a correct bugfix to me.

@Tobion
Copy link
Contributor

Tobion commented Aug 30, 2017

@nicolas-grekas polyfill for bcmath? ;) Not that trivial I assume.

@@ -80,7 +87,7 @@ public function reverseTransform($value)
$value = parent::reverseTransform($value);

if (null !== $value) {
$value *= $this->divisor;
$value = function_exists('bcmul') ? bcmul($value, $this->divisor, $this->scale) : $value * $this->divisor;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes the return value to string (which bcmath works with) which could be a bc break.

@dbrumann
Copy link
Contributor

dbrumann commented Aug 31, 2017

@Tobion I think bcmath-polyfill wouldn't be out of the question. I have seen stuff like adding/multiplying large numbers as code kata before. I think it's more of a question of whether the trade off precision vs. performance is worth it? After all, if I know this issue I can avoid it in other ways in my application which better suit my needs. For example I could be doing (int)(string) $value which is fine in my code where I know that using the divisor will only return floats that should represent a full int value (and where I can probably add comments as to why this weird construct is there), but probably not in Symfony?

edit: But I would certainly be interested in seeing a more generic solution to this. I'm interested to see what @nicolas-grekas or someone else can come up with. 🙂

@@ -68,6 +73,8 @@ public function transform($value)
/**
* Transforms a localized money string into a normalized format.
*
* @requires extension bcmath
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be removed or replaced with @uses.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not needed anymore

@nicolas-grekas
Copy link
Member

I think we can work around the issue by enforcing a precision of 14, which is way enough for a money transformer, and is the minimal precision to represent most float accurately (the max is 17, but we don't care for the potential precision loss at this scale in this specific case).

@linaori
Copy link
Contributor

linaori commented Aug 31, 2017

When working with money/currency and precision is required, the usage of strings and proper libraries should be enforced. The usage of floats/doubles and simply using * and / can lead to extremely annoying bugs as shown in the example.

@nicolas-grekas
Copy link
Member

nicolas-grekas commented Aug 31, 2017

See #21769 for a similar issue that we solved using cast-to-string.
We should do the same here IMHO.

@Rubinum
Copy link
Contributor Author

Rubinum commented Sep 2, 2017

@nicolas-grekas thank you for digging down through the older PRs.
As I understand the older PR that you mentioned, a solution could be to cast the whole operation to string? The return type of reverseTransform and transform would change from int|float to string.


/**
* Transforms a localized money string into a normalized format.
*
* @uses extension bcmath
*
* @param string $value Localized money string
*
* @return string Normalized number
*
* @throws TransformationFailedException If the given value is not a string
*                                       or if the value can not be transformed.
*/
public function reverseTransform($value)
    {
        $value = parent::reverseTransform($value);
        if (null !== $value) {
            $value = (string) ($value * $this->divisor);
        }
        return $value;
    }

/**
* Transforms a localized money string into a normalized format.
*
* @requires extension bcmath
*
* @param string $value Localized money string
*
* @return string Normalized number
*
* @throws TransformationFailedException If the given value is not a string
*                                       or if the value can not be transformed.
*/
public function transform($value)
    {
        if (null !== $value) {
            if (!is_numeric($value)) {
                throw new TransformationFailedException('Expected a numeric.');
            }
            $value = (string) ($value / $this->divisor);
        }
        return parent::transform($value);
    }

I mean we could do it the other way around and cast the string from my solution to float to stick to the return type:

$value = (float) function_exists('bcmul') ? bcmul((string) $value, (string) $this->divisor, $this->scale) : $value * $this->divisor;
/* ... */
$value = (float) function_exists('bcdiv') ? bcdiv((string) $value, (string) $this->divisor, $this->scale) : $value / $this->divisor;

But this looks a bit weird in my opinion ^^. What do you prefer?

@xabbuh
Copy link
Member

xabbuh commented Sep 3, 2017

public function reverseTransform($value)
{
    $value = parent::reverseTransform($value);

    if (null !== $value) {
        $value = (string) ($value * $this->divisor);

        if (ctype_digit($value)) {
            $value = (int) $value;
        } else {
            $value = (float) $value;
        }
    }

    return $value;
}

What about something like this?

@nicolas-grekas nicolas-grekas changed the title Replaced division and multiplication operator by bcmul() and bcdiv(). [Form] Fix precision of MoneyToLocalizedStringTransformer's divisions and multiplications Sep 11, 2017
}

/**
* Transforms a normalized format into a localized money string.
*
* @requires extension bcmath
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's now not true anymore

@@ -23,6 +23,8 @@ class MoneyToLocalizedStringTransformer extends NumberToLocalizedStringTransform
{
private $divisor;

private $scale;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be removed

@@ -40,11 +42,14 @@ public function __construct($scale = 2, $grouping = true, $roundingMode = self::
}

$this->divisor = $divisor;
$this->scale = $scale;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be removed

@Rubinum
Copy link
Contributor Author

Rubinum commented Sep 12, 2017

@smoench. Thank you for that hint. I removed it.

$value = (string) ($value / $this->divisor);

if (ctype_digit($value)) {
$value = (int) $value;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need the integer logic as it doesn't matter for parent::transform whether it's int or float. just cast it to float as it was a float before as well.

$value = (string) ($value * $this->divisor);

if (ctype_digit($value)) {
$value = (int) $value;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it returned a float always due to the multiplikation with the divisor. changing that to int is a behavior change. I'd recommend to just cast to to float always.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be $value = (float) (string) ($value * $this->divisor) okay as solution? @Tobion

Copy link
Contributor Author

@Rubinum Rubinum Sep 13, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe the string convert is not necessary? I tested it and if we always convert to float, everything seems fine. https://3v4l.org/8S9Hg

But we have to change the return typ to float only. Is that a bc?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return value has always been float. The problem is when you cast the return value to int in user-land code. There has never been a "bug" in the class itself. I thought you know what you are trying to fix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am aware of what I am trying to contribute here. Its the same problem as here #21769 (thanks to nicolas-grekas, who linked that PR in his comment).
I will cast the string to float afterwards to fulfill your objection.

@@ -80,7 +86,13 @@ public function reverseTransform($value)
$value = parent::reverseTransform($value);

if (null !== $value) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd propose to change this to if (null !== $value && 1 !== $this-divisor) as the whole divisor logic does not make sense when it's 1 (the default value). please do the same in the transform method as well.

@smoench
Copy link
Contributor

smoench commented Sep 21, 2017

Could this be merged?

@Rubinum Rubinum changed the base branch from master to 2.7 September 26, 2017 15:17
if (!is_numeric($value)) {
throw new TransformationFailedException('Expected a numeric.');
}

$value /= $this->divisor;
$value = (string) ($value / $this->divisor);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks wrong to me. Previously, $value was a float here, but now it is a string.

Copy link
Contributor Author

@Rubinum Rubinum Sep 27, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it from (float) (string) ($value / $this->divisor); to (string) ($value / $this->divisor); because this functions purpose is to Transforms a normalized format into a localized money string. according to the PHPDoc-Block. The return type is also string, thats why I changed it. Should I put a float cast in front of it again?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is fine. the parent transform should be able handle both string or float.

@Tobion
Copy link
Contributor

Tobion commented Sep 29, 2017

Some background: We know floats cannot be represented perfectly so there is some lost precision.
So a float like 36.55 * 100 can actually be 3654.99999999.
The problem when you cast that float to int, PHP cuts the scale and that leaves you with an 3654 instead of 6355. By string casting, PHP will use the number precision and realize 3654.99999999 is just meant to be 3655 and then casting that to float again the float will be represented as something like 3655.00000001 instead which will also cast to the expected integer again.

I wondered how many common montary values are affected by this difference and if it's reliable. Or if there are numbers where the string cast approach actually doesn't work as expected as well. I wrote a little script: https://3v4l.org/JdfjZ

For numbers between 1 and 99999.
Unexpected count traditional approach: 4586
Unexpected count new approach: 0

@Tobion
Copy link
Contributor

Tobion commented Sep 29, 2017

That makes me wonder by PHP by default does not do this automatically, i.e. (int) $float === (int) (string) $float. I guess that would solve some unexpected floating precision with type casting either explicitly or implicitly by scalar type declarations.

@fabpot
Copy link
Member

fabpot commented Sep 29, 2017

Thank you @Rubinum.

@fabpot fabpot merged commit ab47c78 into symfony:2.7 Sep 29, 2017
fabpot added a commit that referenced this pull request Sep 29, 2017
…s divisions and multiplications (Rubinum)

This PR was merged into the 2.7 branch.

Discussion
----------

[Form] Fix precision of MoneyToLocalizedStringTransformer's divisions and multiplications

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | no
| License       | MIT

There is a [PHP Bug](https://bugs.php.net/bug.php?id=75004) with the accuracy of divisions and multiplications when `/=` and `*=` are used.
Here is the proof: https://3v4l.org/u1DkX
It would be better to use `bcmul()` and `bcdiv()` in the `MoneyToLocalizedStringTransformer.php` to prevent this bug.

Commits
-------

ab47c78 Added improvement for accuracy in MoneyToLocalizedStringTransformer.
@Tobion
Copy link
Contributor

Tobion commented Sep 29, 2017

I've found out the behavior also depends on the precision ini setting.

When setting ini_set('precision', 17); or when using the new ini_set('precision', -1); from https://wiki.php.net/rfc/precise_float_value, there is no unexpected behavior.
So for master with php 7.1., we could remove these workarounds and add recommendation to our requirements to use ini_set('precision', -1) which seems much better in most cases. That is the new PHP default anyway, but some users might not make use of it and use old ini values.
For example https://3v4l.org does not use the new ini even for PHP >= 7.1 by default.

@Tobion
Copy link
Contributor

Tobion commented Sep 29, 2017

Actually, this change is not reliable at all what I was fearing the whole time. It only works because the precision is only 14. But if you increase the precision or use PHP 7.1 with more precision by default, (float) (string) $float does not change anything at all. See https://3v4l.org/La5iR

@nicolas-grekas
Copy link
Member

We could for precision to 14 if we want to provide robustness for this (here and in #21769)
(or we an assume nobody changes this "precision" setting and ensure this in check.php)
@Tobion would you do one of the 2 PRs?

@Tobion
Copy link
Contributor

Tobion commented Sep 29, 2017

Neither of that is a proper solution. The only one is using fixed precision calculation with bcmath.

@nicolas-grekas
Copy link
Member

Then bcmath for ppl having it and precision 14 for others? I don't think we can add a strong requirement for bcmath, do you?

@Tobion
Copy link
Contributor

Tobion commented Sep 29, 2017

Using bcmath and precision as fallback seems fine to not break BC. But IMO we should add requirement for bcmath in Symfony 4 so we can remove that ugly workaround. Bcmath is included by default in php so that shouldn't cause much trouble.

@nicolas-grekas
Copy link
Member

It's included really? Cool, so we can trigger a deprecation on 3.4 and throw on 4.0.

This was referenced Oct 5, 2017
fabpot added a commit that referenced this pull request May 17, 2018
…s divisions on transform() (syastrebov)

This PR was merged into the 2.7 branch.

Discussion
----------

[Form] Fix precision of MoneyToLocalizedStringTransformer's divisions on transform()

| Q             | A
| ------------- | ---
| Branch?       | 2.7
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | no
| License       | MIT
| Doc PR        |

Related issue #21026.
Previous PR #24036.
Similar fix for `transform()` method.

Commits
-------

f94b7aa fix rounding from string
symfony-splitter pushed a commit to symfony/form that referenced this pull request May 17, 2018
…s divisions on transform() (syastrebov)

This PR was merged into the 2.7 branch.

Discussion
----------

[Form] Fix precision of MoneyToLocalizedStringTransformer's divisions on transform()

| Q             | A
| ------------- | ---
| Branch?       | 2.7
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | no
| License       | MIT
| Doc PR        |

Related issue symfony/symfony#21026.
Previous PR symfony/symfony#24036.
Similar fix for `transform()` method.

Commits
-------

f94b7aadd3 fix rounding from string
@arthurlauck
Copy link

There is another problem, Im doing a cents to decimal, using MoneyType, so I setted the divisor to 100, but when persisinting an entity, it puts the decimals, like 100.0, thats a problem, when you have the same value, like the entity has 100 and then it tries to update to 100.0 (what is the same thing), but it triggers an useless update that hits the database.

@stof
Copy link
Member

stof commented Nov 10, 2020

@arthurlauck apply the appropriate casting in your entity setter then

@arthurlauck
Copy link

arthurlauck commented Nov 10, 2020

@stof I ended up, extending the MoneyType, would it be better to cast in the entity?

public function buildForm(FormBuilderInterface $builder, array $options)
    {
        $builder->addViewTransformer(new CallbackTransformer(fn($value) => $value, fn($value) => (int) $value));

        parent::buildForm($builder, $options);
    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.