Thanks to visit codestin.com
Credit goes to github.com

Skip to content

np.vectorize fails with output dtype datetime64[ns] #25936

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gcaria opened this issue Mar 5, 2024 · 3 comments
Open

np.vectorize fails with output dtype datetime64[ns] #25936

gcaria opened this issue Mar 5, 2024 · 3 comments
Labels

Comments

@gcaria
Copy link

gcaria commented Mar 5, 2024

Describe the issue:

This came out from @mathause 's investigation in pydata/xarray#8802 (comment)

Reproduce the code example:

import numpy as np
otype = "datetime64[ns]"
arr = np.array(['2024-01-01', '2024-01-02', '2024-01-03'], dtype='datetime64[ns]')
np.vectorize(lambda x: x, signature="(i)->(j)", otypes=[otype])(arr)

Error message:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 4
      2 otype = "datetime64[ns]"
      3 arr = np.array(['2024-01-01', '2024-01-02', '2024-01-03'], dtype='datetime64[ns]')
----> 4 np.vectorize(lambda x: x, signature="(i)->(j)", otypes=[otype])(arr)

File ~/envs/test-xarray/lib/python3.10/site-packages/numpy/lib/function_base.py:2372, in vectorize.__call__(self, *args, **kwargs)
   2369     self._init_stage_2(*args, **kwargs)
   2370     return self
-> 2372 return self._call_as_normal(*args, **kwargs)

File ~/envs/test-xarray/lib/python3.10/site-packages/numpy/lib/function_base.py:2365, in vectorize._call_as_normal(self, *args, **kwargs)
   2362     vargs = [args[_i] for _i in inds]
   2363     vargs.extend([kwargs[_n] for _n in names])
-> 2365 return self._vectorize_call(func=func, args=vargs)

File ~/envs/test-xarray/lib/python3.10/site-packages/numpy/lib/function_base.py:2446, in vectorize._vectorize_call(self, func, args)
   2444 """Vectorized call to `func` over positional `args`."""
   2445 if self.signature is not None:
-> 2446     res = self._vectorize_call_with_signature(func, args)
   2447 elif not args:
   2448     res = func()

File ~/envs/test-xarray/lib/python3.10/site-packages/numpy/lib/function_base.py:2506, in vectorize._vectorize_call_with_signature(self, func, args)
   2502         outputs = _create_arrays(broadcast_shape, dim_sizes,
   2503                                  output_core_dims, otypes, results)
   2505     for output, result in zip(outputs, results):
-> 2506         output[index] = result
   2508 if outputs is None:
   2509     # did not call the function even once
   2510     if otypes is None:

ValueError: Cannot convert from specific units to generic units in NumPy datetimes or timedeltas

Python and NumPy Versions:

1.26.4
3.10.1 (main, Dec 15 2021, 17:45:54) [GCC 9.3.0]

Runtime Environment:

[{'numpy_version': '1.26.4',
  'python': '3.10.1 (main, Dec 15 2021, 17:45:54) [GCC 9.3.0]',
  'uname': uname_result(system='Linux', node='ip-172-34-24-37', release='5.15.0-1051-aws', version='#56~20.04.1-Ubuntu SMP Tue Nov 28 15:43:31 UTC 2023', machine='x86_64')},
 {'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
                      'found': ['SSSE3',
                                'SSE41',
                                'POPCNT',
                                'SSE42',
                                'AVX',
                                'F16C',
                                'FMA3',
                                'AVX2',
                                'AVX512F',
                                'AVX512CD',
                                'AVX512_SKX'],
                      'not_found': ['AVX512_KNL',
                                    'AVX512_KNM',
                                    'AVX512_CLX',
                                    'AVX512_CNL',
                                    'AVX512_ICL']}},
 {'architecture': 'SkylakeX',
  'filepath': '/home/ubuntu/envs/test-xarray/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so',
  'internal_api': 'openblas',
  'num_threads': 4,
  'prefix': 'libopenblas',
  'threading_layer': 'pthreads',
  'user_api': 'blas',
  'version': '0.3.23.dev'}]
None

Context for the issue:

No response

@mathause
Copy link

mathause commented Mar 5, 2024

I think this happens because numpy creates the target array with np.dtype("datetime64[ns]").char, i.e. dtype('<M8') and not dtype('<M8[ms]')

otypes = ''.join([_nx.dtype(x).char for x in otypes])

@ngoldbaum
Copy link
Member

ngoldbaum commented Mar 5, 2024

I wonder what breaks if you drop the char there. Looking at the Vectorize class, it seems to only pass otypes entries as values for dtype keywords, so dtype instances should be fine.

@batatas-fritas
Copy link
Contributor

I will work on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants