Thanks to visit codestin.com
Credit goes to github.com

Skip to content

floating point errors #9237

@todbot

Description

@todbot

CircuitPython version

Adafruit CircuitPython 9.1.0-beta.1-18-g781c577745 on 2024-05-05; Adafruit QT Py RP2040 with rp2040

Code/REPL

print("Hello, Pi Pico!")

print("0.004 = ", 0.004)
print("1000 = ", 1000)
print("0.004 * 1000 = ", 0.004 * 1000)
print("int(0.004 * 1000) = ", int(0.004 * 1000))
print('"%f" % (0.004 * 1000) = ', "%f" % (0.004 * 1000))

Behavior

Prints out a "3" instead of a "4" on the second-to-last line and "3.999998" instead of "4.000000"

Hello, Pi Pico!
0.004 =  0.004
1000 =  1000
0.004 * 1000 =  4.0
int(0.004 * 1000) =  3
"%f" % (0.004 * 1000) =  3.999998

Description

The crux of the issue is that int(0.004 * 1000) prints out "3" instead of "4".

It's unclear to me why it does this, both numbers have pretty simple representations in IEEE754 (using https://www.h-schmidt.net/FloatConverter/IEEE754.html, 0.004 is actually 0.0040000001899898052215576171875, 1000 is actually 1000)

This error does not exist in MicroPython v1.22.0 on 2023-12-27.

Here are two wokwis to compare:

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions