Thanks to visit codestin.com
Credit goes to github.com

Skip to content

np.prod returns 0 in case of integer overflow (Trac #1629) #2225

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
numpy-gitbot opened this issue Oct 19, 2012 · 5 comments
Closed

np.prod returns 0 in case of integer overflow (Trac #1629) #2225

numpy-gitbot opened this issue Oct 19, 2012 · 5 comments
Labels
00 - Bug component: Other
Milestone
1.5.1

Comments

@numpy-gitbot
Copy link

Original ticket http://projects.scipy.org/numpy/ticket/1629 on 2010-10-10 by trac user kilian, assigned to unknown.

I am using np.prod to calculate the number of elements of a sparse matrix (np.prod(C.shape)) and I noticed the following behavior: In case the result is greater than 2**31, zero is returned. Instead, the result should be converted to int long int (or at least an exception should be raised).

>>> import numpy as np
>>> np.version.version
'1.4.0'
>>> np.prod([2**16,2**16])
0
@numpy-gitbot
Copy link
Author

@charris wrote on 2010-10-10

I don't see this for current svn on my 64 bit system:

In [3]: array([2**16,2**16]).prod()
Out[3]: 4294967296

In [4]: array([2**16,2**16]).prod(dtype=int64)
Out[4]: 4294967296

In [7]: array([2**16,2**16]).dtype
Out[7]: dtype('int64')

However, it could be that the default accumator on your system is int32? What platform are you on? In any case, you should specify the type of the result, numpy can't tell in advance how big the product is going to be and it could easily be out of range for anything but arbitrary precision integers.

@numpy-gitbot
Copy link
Author

@rgommers wrote on 2010-10-10

It does depend on whether the platform is 32/64-bit:

In [10]: np.prod([2**16,2**16])
Out[10]: 0

In [11]: np.prod([2**16,2**16]).dtype
Out[11]: dtype('int32')

But the documentation explains this issue in detail, quoting relevant parts:

    Parameters
    ----------
...
    dtype : data-type, optional
        The data-type of the returned array, as well as of the accumulator
        in which the elements are multiplied.  By default, if `a` is of
        integer type, `dtype` is the default platform integer. (Note: if
        the type of `a` is unsigned, then so is `dtype`.)  Otherwise,
        the dtype is the same as that of `a`.
...
    Notes
    -----
    Arithmetic is modular when using integer types, and no error is
    raised on overflow.  That means that, on a 32-bit platform:

    >>> x = np.array([536870910, 536870910, 536870910, 536870910])
    >>> np.prod(x) #random
    16

There probably was a good reason to implement and document it like this, but that's best asked on the mailing list. I think this ticket can be closed.

@numpy-gitbot
Copy link
Author

@pv wrote on 2010-10-10

The good reason is that the alternative is checking integer overflow manually, and that the output type of an ufunc should depend only on the input type, not on the input values.

@numpy-gitbot
Copy link
Author

Milestone changed to 1.5.1 by @pv on 2010-10-10

@numpy-gitbot
Copy link
Author

trac user kilian wrote on 2010-10-11

Ok, thanks for all the comments! I should have read the doc string more carefully first...

HaoZeke added a commit to HaoZeke/numpy that referenced this issue Aug 25, 2023
@HaoZeke @NamamiShanker
ENH: Add meson skeleton from numpygh-2225
cfe6a11 
Co-authored-by: NamamiShanker <[email protected]>
HaoZeke added a commit to HaoZeke/numpy that referenced this issue Aug 25, 2023
@HaoZeke @NamamiShanker
ENH: Add meson skeleton from numpygh-2225
873a8bd 
Co-authored-by: NamamiShanker <[email protected]>
HaoZeke added a commit to HaoZeke/numpy that referenced this issue Sep 2, 2023
@HaoZeke @NamamiShanker
ENH: Add meson skeleton from numpygh-2225
a0b01c3 
Co-authored-by: NamamiShanker <[email protected]>
charris pushed a commit that referenced this issue Sep 5, 2023
@HaoZeke @NamamiShanker @rgommers
ENH: meson backend for f2py (#24532)
fedc834 
* FIX: Import f2py2e rather than f2py for run_main

* FIX: Import f2py2e instead of f2py

* ENH: Add F2PY back-end work from gh-22225

Co-authored-by: NamamiShanker <[email protected]>

* ENH: Add meson skeleton from gh-2225

Co-authored-by: NamamiShanker <[email protected]>

* MAINT: Trim backend.py down to f2py2e flags

* ENH: Add a factory function for backends

* ENH: Add a distutils backend

* ENH: Handle --backends in f2py

Defaults to distutils for now

* DOC: Add some minor comments in f2py2e

* MAINT: Refactor and rework meson.build.src

* MAINT: Add objects

* MAINT: Cleanup distutils backend

* MAINT: Refactor to add everything back to backend

Necessary for the meson.build for now. Refactors / cleanup needs better
argument handling in f2py2e

* MAINT: Fix overly long line

* BUG: Construct wrappers for meson backend

* MAINT: Rework, simplify template massively

* ENH: Truncate meson.build to skeleton only

* MAINT: Minor backend housekeeping, name changes

* MAINT: Less absolute paths, update setup.py [f2py]

* MAINT: Move f2py module name functionality

Previously part of np.distutils

* ENH: Handle .pyf files

* TST: Fix typo in isoFortranEnvMap.f90

* MAINT: Typo in f2py2e support for pyf files

* DOC: Add release note for --backend

* MAINT: Conditional switch for Python 3.12 [f2py]

* MAINT: No absolute paths in backend [f2py-meson]

The files are copied over anyway, this makes it easier to extend the
generated skeleton

* MAINT: Prettier generated meson.build files [f2py]

* ENH: Add meson's dependency(blah) to f2py

* DOC: Document the new flag

* MAINT: Simplify and rename backend template [f2py]

Co-authored-by: rgommers <[email protected]>

* ENH: Support build_type via --debug [f2py-meson]

* MAINT,DOC: Reduce warn,rework doc [f2py-meson]

Co-authored-by: rgommers <[email protected]>

* ENH: Rework deps: to --dep calls [f2py-meson]

Also shows how incremental updates to the parser can be done.

* MAINT,DOC: Add --backend to argparse, add docs

* MAINT: Rename meson template [f2py-meson]

* MAINT: Add meson.build for f2py

Should address #22225 (comment)

* BLD: remove duplicate f2py handling in meson.build files

---------

Co-authored-by: Namami Shanker <[email protected]>
Co-authored-by: NamamiShanker <[email protected]>
Co-authored-by: rgommers <[email protected]>
Co-authored-by: Ralf Gommers <[email protected]>
charris pushed a commit to charris/numpy that referenced this issue Sep 5, 2023
@HaoZeke @NamamiShanker
@rgommers @charris
ENH: meson backend for f2py (numpy#24532)
4fe7ea3 
* FIX: Import f2py2e rather than f2py for run_main

* FIX: Import f2py2e instead of f2py

* ENH: Add F2PY back-end work from numpygh-22225

Co-authored-by: NamamiShanker <[email protected]>

* ENH: Add meson skeleton from numpygh-2225

Co-authored-by: NamamiShanker <[email protected]>

* MAINT: Trim backend.py down to f2py2e flags

* ENH: Add a factory function for backends

* ENH: Add a distutils backend

* ENH: Handle --backends in f2py

Defaults to distutils for now

* DOC: Add some minor comments in f2py2e

* MAINT: Refactor and rework meson.build.src

* MAINT: Add objects

* MAINT: Cleanup distutils backend

* MAINT: Refactor to add everything back to backend

Necessary for the meson.build for now. Refactors / cleanup needs better
argument handling in f2py2e

* MAINT: Fix overly long line

* BUG: Construct wrappers for meson backend

* MAINT: Rework, simplify template massively

* ENH: Truncate meson.build to skeleton only

* MAINT: Minor backend housekeeping, name changes

* MAINT: Less absolute paths, update setup.py [f2py]

* MAINT: Move f2py module name functionality

Previously part of np.distutils

* ENH: Handle .pyf files

* TST: Fix typo in isoFortranEnvMap.f90

* MAINT: Typo in f2py2e support for pyf files

* DOC: Add release note for --backend

* MAINT: Conditional switch for Python 3.12 [f2py]

* MAINT: No absolute paths in backend [f2py-meson]

The files are copied over anyway, this makes it easier to extend the
generated skeleton

* MAINT: Prettier generated meson.build files [f2py]

* ENH: Add meson's dependency(blah) to f2py

* DOC: Document the new flag

* MAINT: Simplify and rename backend template [f2py]

Co-authored-by: rgommers <[email protected]>

* ENH: Support build_type via --debug [f2py-meson]

* MAINT,DOC: Reduce warn,rework doc [f2py-meson]

Co-authored-by: rgommers <[email protected]>

* ENH: Rework deps: to --dep calls [f2py-meson]

Also shows how incremental updates to the parser can be done.

* MAINT,DOC: Add --backend to argparse, add docs

* MAINT: Rename meson template [f2py-meson]

* MAINT: Add meson.build for f2py

Should address numpy#22225 (comment)

* BLD: remove duplicate f2py handling in meson.build files

---------

Co-authored-by: Namami Shanker <[email protected]>
Co-authored-by: NamamiShanker <[email protected]>
Co-authored-by: rgommers <[email protected]>
Co-authored-by: Ralf Gommers <[email protected]>
@charris charris mentioned this issue Sep 5, 2023
Merged
charris added a commit that referenced this issue Sep 5, 2023
@charris @HaoZeke
@NamamiShanker @rgommers
Merge pull request #24647 from charris/backport-24532
acde96a 
* FIX: Import f2py2e rather than f2py for run_main

* FIX: Import f2py2e instead of f2py

* ENH: Add F2PY back-end work from gh-22225



* ENH: Add meson skeleton from gh-2225



* MAINT: Trim backend.py down to f2py2e flags

* ENH: Add a factory function for backends

* ENH: Add a distutils backend

* ENH: Handle --backends in f2py

Defaults to distutils for now

* DOC: Add some minor comments in f2py2e

* MAINT: Refactor and rework meson.build.src

* MAINT: Add objects

* MAINT: Cleanup distutils backend

* MAINT: Refactor to add everything back to backend

Necessary for the meson.build for now. Refactors / cleanup needs better
argument handling in f2py2e

* MAINT: Fix overly long line

* BUG: Construct wrappers for meson backend

* MAINT: Rework, simplify template massively

* ENH: Truncate meson.build to skeleton only

* MAINT: Minor backend housekeeping, name changes

* MAINT: Less absolute paths, update setup.py [f2py]

* MAINT: Move f2py module name functionality

Previously part of np.distutils

* ENH: Handle .pyf files

* TST: Fix typo in isoFortranEnvMap.f90

* MAINT: Typo in f2py2e support for pyf files

* DOC: Add release note for --backend

* MAINT: Conditional switch for Python 3.12 [f2py]

* MAINT: No absolute paths in backend [f2py-meson]

The files are copied over anyway, this makes it easier to extend the
generated skeleton

* MAINT: Prettier generated meson.build files [f2py]

* ENH: Add meson's dependency(blah) to f2py

* DOC: Document the new flag

* MAINT: Simplify and rename backend template [f2py]



* ENH: Support build_type via --debug [f2py-meson]

* MAINT,DOC: Reduce warn,rework doc [f2py-meson]



* ENH: Rework deps: to --dep calls [f2py-meson]

Also shows how incremental updates to the parser can be done.

* MAINT,DOC: Add --backend to argparse, add docs

* MAINT: Rename meson template [f2py-meson]

* MAINT: Add meson.build for f2py

Should address #22225 (comment)

* BLD: remove duplicate f2py handling in meson.build files

---------

Co-authored-by: Rohit Goswami <[email protected]>
Co-authored-by: Namami Shanker <[email protected]>
Co-authored-by: NamamiShanker <[email protected]>
Co-authored-by: rgommers <[email protected]>
Co-authored-by: Ralf Gommers <[email protected]>
charris pushed a commit to charris/numpy that referenced this issue Nov 11, 2023
@HaoZeke @NamamiShanker
@rgommers @charris
ENH: meson backend for f2py (numpy#24532)
174e45a 
* FIX: Import f2py2e rather than f2py for run_main

* FIX: Import f2py2e instead of f2py

* ENH: Add F2PY back-end work from numpygh-22225

Co-authored-by: NamamiShanker <[email protected]>

* ENH: Add meson skeleton from numpygh-2225

Co-authored-by: NamamiShanker <[email protected]>

* MAINT: Trim backend.py down to f2py2e flags

* ENH: Add a factory function for backends

* ENH: Add a distutils backend

* ENH: Handle --backends in f2py

Defaults to distutils for now

* DOC: Add some minor comments in f2py2e

* MAINT: Refactor and rework meson.build.src

* MAINT: Add objects

* MAINT: Cleanup distutils backend

* MAINT: Refactor to add everything back to backend

Necessary for the meson.build for now. Refactors / cleanup needs better
argument handling in f2py2e

* MAINT: Fix overly long line

* BUG: Construct wrappers for meson backend

* MAINT: Rework, simplify template massively

* ENH: Truncate meson.build to skeleton only

* MAINT: Minor backend housekeeping, name changes

* MAINT: Less absolute paths, update setup.py [f2py]

* MAINT: Move f2py module name functionality

Previously part of np.distutils

* ENH: Handle .pyf files

* TST: Fix typo in isoFortranEnvMap.f90

* MAINT: Typo in f2py2e support for pyf files

* DOC: Add release note for --backend

* MAINT: Conditional switch for Python 3.12 [f2py]

* MAINT: No absolute paths in backend [f2py-meson]

The files are copied over anyway, this makes it easier to extend the
generated skeleton

* MAINT: Prettier generated meson.build files [f2py]

* ENH: Add meson's dependency(blah) to f2py

* DOC: Document the new flag

* MAINT: Simplify and rename backend template [f2py]

Co-authored-by: rgommers <[email protected]>

* ENH: Support build_type via --debug [f2py-meson]

* MAINT,DOC: Reduce warn,rework doc [f2py-meson]

Co-authored-by: rgommers <[email protected]>

* ENH: Rework deps: to --dep calls [f2py-meson]

Also shows how incremental updates to the parser can be done.

* MAINT,DOC: Add --backend to argparse, add docs

* MAINT: Rename meson template [f2py-meson]

* MAINT: Add meson.build for f2py

Should address numpy#22225 (comment)

* BLD: remove duplicate f2py handling in meson.build files

---------

Co-authored-by: Namami Shanker <[email protected]>
Co-authored-by: NamamiShanker <[email protected]>
Co-authored-by: rgommers <[email protected]>
Co-authored-by: Ralf Gommers <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug component: Other
Projects
None yet
Milestone
1.5.1
Development

No branches or pull requests
1 participant
@numpy-gitbot