tree-wide: fix implicit casts #5773

dlech · 2020-03-20T05:16:40Z

These were found by building the unix coverage variant on macOS (so clang compiler). Mostly, these are fixing implicit cast of float/double to mp_float_t which is one of those two and one mp_int_t to size_t fix for good measure.

dpgeorge

Would it be worth building the coverage build in the Travis osx job?

dpgeorge · 2020-03-20T05:53:33Z

ports/unix/modtime.c

@@ -61,7 +61,7 @@ static inline int msec_sleep_tv(struct timeval *tv) {
 #endif

 #if defined(MP_CLOCKS_PER_SEC)
-#define CLOCK_DIV (MP_CLOCKS_PER_SEC / 1000.0F)
+#define CLOCK_DIV (MP_CLOCKS_PER_SEC / (mp_float_t)1000.0)


there's a macro MICROPY_FLOAT_CONST(x) which might be better suited here

dpgeorge · 2020-03-20T05:55:26Z

py/objarray.c

@@ -430,7 +430,7 @@ STATIC mp_obj_t array_subscr(mp_obj_t self_in, mp_obj_t index_in, mp_obj_t value
                }

                // TODO: check src/dst compat
-                mp_int_t len_adj = src_len - (slice.stop - slice.start);
+                size_t len_adj = src_len - (slice.stop - slice.start);


I think len_adj can be negative

dpgeorge · 2020-03-20T05:55:44Z

py/parsenum.c

@@ -218,7 +218,7 @@ mp_obj_t mp_parse_num_decimal(const char *str, size_t len, bool allow_imag, bool
        if (str + 2 < top && (str[1] | 0x20) == 'n' && (str[2] | 0x20) == 'f') {
            // inf
            str += 3;
-            dec_val = INFINITY;
+            dec_val = (mp_float_t)INFINITY;


probably also use MICROPY_FLOAT_CONST(x) here

This doesn't work since MICROPY_FLOAT_CONST either passes the value through untouched or adds an F. So if mp_float_t is double, we still get a compile error or if mp_float_t is float, then we get undefined INFINITYF.

stinos · 2020-03-20T07:33:04Z

IIRC there's more of these, and also a lot of implicit signed/unsigned casts. At least I remember that building with the default warning level with msvc showed more warnings than what gets fixed here. Will get back later today with a list

stinos · 2020-03-20T08:50:36Z

Ok so there's these:

'=': conversion from 'double' to 'long', possible loss of data  ports\unix\modtime.c  99  
'=': conversion from 'double' to 'long', possible loss of data  ports\unix\modtime.c  100 
'=': conversion from 'mpz_dig_t' to 'byte', possible loss of data py\mpz.c  1632  
'=': conversion from 'mpz_dig_t' to 'byte', possible loss of data py\mpz.c  1637  
'=': conversion from 'mp_float_t' to 'float', possible loss of data py\binary.c 314 
'=': conversion from 'mp_float_t' to 'float', possible loss of data py\binary.c 362 
'=': conversion from 'mp_float_t' to 'float', possible loss of data extmod\moductypes.c 378 
'=': conversion from 'mp_float_uint_t' to 'size_t', possible loss of data py\mpz.c  783 
'=': conversion from 'mp_int_t' to 'float', possible loss of data py\binary.c 422 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\emitbc.c 132 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\emitbc.c 133 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\emitbc.c 239 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\emitbc.c 240 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\qstr.c 250 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\persistentcode.c 307 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\persistentcode.c 308 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\persistentcode.c 310 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\persistentcode.c 311 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\persistentcode.c 340 
'=': conversion from 'size_t' to 'byte', possible loss of data  py\persistentcode.c 341 
'=': conversion from 'size_t' to 'uint16_t', possible loss of data  py\lexer.c  159 
'=': conversion from 'size_t' to 'uint16_t', possible loss of data  py\objgenerator.c 69  
'=': conversion from 'size_t' to 'uint16_t', possible loss of data  py\objfun.c 279 
'=': conversion from 'size_t' to 'uint16_t', possible loss of data  py\objnamedtuple.c  158 
'=': conversion from 'size_t' to 'uint16_t', possible loss of data  py\objtype.c  1132  
'=': conversion from 'size_t' to 'uint16_t', possible loss of data  py\persistentcode.c 76  
'=': conversion from 'size_t' to 'uint16_t', possible loss of data  py\persistentcode.c 88  
'=': conversion from 'size_t' to 'uint16_t', possible loss of data  py\scope.c  47  
'=': conversion from 'uint64_t' to 'mp_uint_t', possible loss of data  py\binary.c 325 
'=': conversion from 'unsigned __int64' to 'BYTE', possible loss of data  extmod\crypto-algorithms\sha256.c 136 
'=': conversion from 'unsigned __int64' to 'BYTE', possible loss of data  extmod\crypto-algorithms\sha256.c 137 
'=': conversion from 'unsigned __int64' to 'BYTE', possible loss of data  extmod\crypto-algorithms\sha256.c 138 
'=': conversion from 'unsigned __int64' to 'BYTE', possible loss of data  extmod\crypto-algorithms\sha256.c 139 
'=': conversion from 'unsigned __int64' to 'BYTE', possible loss of data  extmod\crypto-algorithms\sha256.c 140 
'=': conversion from 'unsigned __int64' to 'BYTE', possible loss of data  extmod\crypto-algorithms\sha256.c 141 
'=': conversion from 'unsigned __int64' to 'BYTE', possible loss of data  extmod\crypto-algorithms\sha256.c 142 
'=': conversion from 'unsigned __int64' to 'long', possible loss of data  ports\windows\msvc\gettimeofday.c 51  
'=': conversion from 'unsigned __int64' to 'long', possible loss of data  ports\windows\msvc\gettimeofday.c 52  
'=': conversion from '__int64' to 'mp_uint_t', possible loss of data  extmod\moductypes.c 462 
'>': signed/unsigned mismatch py\objarray.c 448 
'function': conversion from 'size_t' to 'char', possible loss of data py\objarray.c 86  
'function': conversion from 'size_t' to 'char', possible loss of data py\objarray.c 338 
'function': conversion from 'size_t' to 'char', possible loss of data py\objarray.c 344 
'function': conversion from 'size_t' to 'char', possible loss of data py\objarray.c 362 
'function': conversion from 'size_t' to 'char', possible loss of data py\objarray.c 484 
'function': conversion from 'size_t' to 'uint8_t', possible loss of data  py\parse.c  312 
'function': conversion from 'size_t' to 'uint8_t', possible loss of data  py\parse.c  864 
'function': conversion from 'time_t' to 'mp_int_t', possible loss of data ports\unix\modtime.c  191 
'function': conversion from 'uint16_t' to 'byte', possible loss of data py\emitbc.c 410 
'initializing': conversion from 'size_t' to 'uint8_t', possible loss of data  py\parse.c  317 
'initializing': conversion from '__int64' to 'uint32_t', possible loss of data  py\binary.c 246

Not sure if these should all be fixed. If so let me know. The ones concerning float/int would best be added to this commit so everything is done in one go?

If all of the above gets fixed there's only some minor warnings left which should probably be disabled on the compiler and not fixed (mainly 'unary minus operator applied to unsigned type, result still unsigned' like for instance in py\emitbc.c line 724' which is quite uniteresting), and then the msvc port can be built with warning level 3 instead of 1 which would be good overall.

dlech · 2020-03-21T23:30:50Z

Would it be worth building the coverage build in the Travis osx job?

I suppose if we want to catch compile errors like this

dlech · 2020-03-22T22:43:44Z

It looks like the test failure is for thread_stacksize1 which has been failing intermittently on other PRs as well.

dpgeorge · 2020-03-24T14:29:25Z

Not sure if these should all be fixed. If so let me know. The ones concerning float/int would best be added to this commit so everything is done in one go?

@stinos I'm happy to have these fixed, but each one would need to be looked over to see exactly how to fix it (probably just cast up, but in some cases need to be careful with sign).

I think all the float-float implicit casts are fixed by this PR.

Would be good if the warnings could be gotten on *nix by enabling some compiler flags... is that possible?

dpgeorge · 2020-03-24T14:30:08Z

py/objarray.c

@@ -430,7 +430,7 @@ STATIC mp_obj_t array_subscr(mp_obj_t self_in, mp_obj_t index_in, mp_obj_t value
                }

                // TODO: check src/dst compat
-                mp_int_t len_adj = src_len - (slice.stop - slice.start);
+                long len_adj = src_len - (slice.stop - slice.start);


I'm not sure making it long is the right thing to do... it's not obvious what it should be but how about just keeping it as mp_int_t because (so far) that seems to work? Otherwise I'd suggest using ssize_t.

I tried ssize_t, but that is a posix type, not a standard C type, so it causes issues on some ports with the type not being defined. So yes, we can keep it as mp_int_t, but will have to add more casts elsewhere.

I tested this out building the coverage build with clang on Linux and I get the same warnings, and can fix this one simply by adding the size_t cast below, I didn't need to change the type here (it could remain mp_int_t). So I'd prefer that, to not change this type.

stinos · 2020-03-25T12:49:12Z

Would be good if the warnings could be gotten on *nix by enabling some compiler flags... is that possible?

It's a bit peculiar it seems. I started with Wfloat-conversion and this warns for a bunch of things msvc also warns about, e.g.

../../py/binary.c: In function ‘mp_binary_set_val’:
../../py/binary.c:314:23: warning: conversion to ‘float’ from ‘mp_float_t {aka double}’ may alter its value [-Wfloat-conversion]
             fp_sp.f = mp_obj_get_float(val_in);
                       ^~~~~~~~~~~~~~~~
../../py/binary.c: In function ‘mp_binary_set_val_array’:
../../py/binary.c:362:35: warning: conversion to ‘float’ from ‘mp_float_t {aka double}’ may alter its value [-Wfloat-conversion]
             ((float *)p)[index] = mp_obj_get_float(val_in);
                                   ^~~~~~~~~~~~~~~~

Since both compilers agree this would be a good candidate for fixing; I'm not sure what it takes to build with clang on unix so didn't try that, but e.g. the above cases are not fixed in this PR. @dlech is this a 32-bit build perhaps, or any idea what goes on?

Then I moved on to -Wconversion -Wno-sign-conversion but that produces hundreds of warnings. Most of those are from btree and axtls though. Is there an easy way to tell the makefile to only apply CFLAGS_EXTRA only to PY_CORE_O so it's easier to start with?

dlech · 2020-03-25T14:30:19Z

@dlech is this a 32-bit build perhaps, or any idea what goes on?

This PR is based off errors from 64-bit clang on macOS 10.15.

To build with clang on Linux, just add CC=clang to the make options. (assuming you have already installed clang of course).

stinos · 2020-03-25T15:34:48Z

64-bit clang

So mp_float_t is double, right?

Will try with clang later. I just don't understand why it would warn for e.g. mp_obj_new_float(x) when x is float. If mp_float_t is float there's no conversion, otherwise it's from float to double which should not cause a (considerable) loss? And on the other hand it does not warn for the opposite direction. Strange.

stinos · 2020-03-25T16:05:05Z

Nevermid, after seeing actual warning I understood (please add this to the commit message so it's clear what is being fixed here and why). Full warning is

warning: implicit conversion increases floating-point precision: 'float' to 'mp_float_t' (aka 'double') [-Wdouble-promotion]

I'm actually not convinced this needs fixing with casts: idea behind this flag is to show you where you're using doubles on targets which don't have FPU support. I.e. just casting doesn't fix much, but does make the use of Wdouble-promotion useless. Removing the flag seems the better option then.

doc:

Give a warning when a value of type float is implicitly promoted to double. CPUs with a 32-bit “single-precision” floating-point unit implement float in hardware, but emulate double in software. On such a machine, doing computations using double values is much more expensive because of the overhead required for software emulation.

dpgeorge · 2020-03-27T03:57:18Z

Is there an easy way to tell the makefile to only apply CFLAGS_EXTRA only to PY_CORE_O so it's easier to start with?

@stinos yes, grep for CSUPEROPT in py/py.mk.

dpgeorge · 2020-03-28T12:12:39Z

Can you please rebase on latest master because there were changes to .travis.yml which now clash here: some uasyncio tests are excluded from the osx job. Simplest way around this would be to just add a build of the coverage variant on osx and not run its tests.

These were found by buiding the unix coverage variant on macOS (so clang compiler). Mostly, these are fixing implicit cast of float/double to mp_float_t which is one of those two and one mp_int_t to size_t fix for good measure.

dlech · 2020-03-28T16:38:44Z

rebased and changed long back to mp_int_t.

This should help catch more compile errors with the clang compiler.

dpgeorge · 2020-03-30T01:08:52Z

Thanks a lot, merged in a2110bd and 2461349

stinos · 2020-03-30T08:12:25Z

Sorry but would really have been better if the commit message said what casts exactly get fixed here, because it's not all of them, nor the ones which actually loose precision :[

dpgeorge · 2020-03-30T13:00:03Z

Sorry but would really have been better if the commit message said what casts exactly get fixed here, because it's not all of them, nor the ones which actually loose precision

@stinos sorry it got a bit confusing with this and #5805, and I wanted to get this in before moving on to that PR. The changes here fix compilation under clang, and (as you point out) fix -Wdouble-promotion issues. So that seemed self contained to me. But yes it would have been better to describe this better in the commit. In fact, may have been better to just fix the -Wdouble-promotion on its own: some of the changes here were for symmetry (ie loss of precision rather than promotion) in code like in binary.c where it handles both "f" and "d" type codes.

If you think some of the changes should be reverted, let me know.

stinos · 2020-03-30T13:17:42Z

Well I think it would be ideal to have the situation as you describe (just the float->double promotion fixes in one commit without the changes for symmetry), and then go on and fix other things. But I'm not sure if it's worth messing up git history a bit just for that, your call I'd say.

dpgeorge · 2020-03-30T13:47:55Z

Well I think it would be ideal to have the situation as you describe (just the float->double promotion fixes in one commit without the changes for symmetry), and then go on and fix other things. But I'm not sure if it's worth messing up git history a bit just for that, your call I'd say.

Maybe we will arrive at a different solution, depending on what everyone thinks about mp_obj_new_float_from_f() etc: #5805 (comment)

…ries-2021-12-23-7.1.x Update frozen libraries in preparation for 7.1.0-rc.0

dpgeorge reviewed Mar 20, 2020

View reviewed changes

dlech force-pushed the implicit-cast branch 5 times, most recently from 6b5f22e to 00df509 Compare March 22, 2020 22:22

dlech mentioned this pull request Mar 23, 2020

tests/thread/thread_stacksize1: Fix crash on certain platforms #5784

Closed

dpgeorge reviewed Mar 24, 2020

View reviewed changes

stinos mentioned this pull request Mar 26, 2020

Fix warnings from -Wfloat-conversion #5805

Closed

tree-wide: Fix implicit casts

dac991f

These were found by buiding the unix coverage variant on macOS (so clang compiler). Mostly, these are fixing implicit cast of float/double to mp_float_t which is one of those two and one mp_int_t to size_t fix for good measure.

dlech force-pushed the implicit-cast branch from 00df509 to 3cca5d8 Compare March 28, 2020 16:37

travis: Build full coverage tests on macOS

b966883

This should help catch more compile errors with the clang compiler.

dlech force-pushed the implicit-cast branch from 3cca5d8 to b966883 Compare March 28, 2020 16:43

dpgeorge closed this Mar 30, 2020

dlech deleted the implicit-cast branch March 30, 2020 14:52

tannewt pushed a commit to tannewt/circuitpython that referenced this pull request Dec 29, 2021

Merge pull request micropython#5773 from dhalbert/update-frozen-libra…

35f0330

…ries-2021-12-23-7.1.x Update frozen libraries in preparation for 7.1.0-rc.0

Uh oh!

tree-wide: fix implicit casts #5773

tree-wide: fix implicit casts #5773

Uh oh!

Conversation

dlech commented Mar 20, 2020

Uh oh!

dpgeorge left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stinos commented Mar 20, 2020

Uh oh!

stinos commented Mar 20, 2020

Uh oh!

dlech commented Mar 21, 2020

Uh oh!

dlech commented Mar 22, 2020

Uh oh!

dpgeorge commented Mar 24, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stinos commented Mar 25, 2020

Uh oh!

dlech commented Mar 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stinos commented Mar 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stinos commented Mar 25, 2020

Uh oh!

dpgeorge commented Mar 27, 2020

Uh oh!

dpgeorge commented Mar 28, 2020

Uh oh!

dlech commented Mar 28, 2020

Uh oh!

dpgeorge commented Mar 30, 2020

Uh oh!

stinos commented Mar 30, 2020

Uh oh!

dpgeorge commented Mar 30, 2020

Uh oh!

stinos commented Mar 30, 2020

Uh oh!

dpgeorge commented Mar 30, 2020

Uh oh!

Uh oh!

dlech commented Mar 25, 2020 •

edited

Loading

stinos commented Mar 25, 2020 •

edited

Loading