Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Comparison ops for Complex Tensors #36444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
anjali411 opened this issue Apr 12, 2020 · 6 comments
Open

Comparison ops for Complex Tensors #36444

anjali411 opened this issue Apr 12, 2020 · 6 comments
Labels
module: complex Related to complex number support in PyTorch module: numpy Related to numpy support, and also numpy compatibility of our operators triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@anjali411
Copy link
Contributor

anjali411 commented Apr 12, 2020

This issue is intended to roll-up several conversations about how PyTorch should compare complex values and how functions that logically rely on comparisons, like min, max, sort, and clamp, should work when given complex inputs. See #36374, which discussed complex min and max, and #33568, which discussed complex clamp. The challenge of comparing complex numbers is not limited to PyTorch, either, see numpy/numpy#15630 for NumPy's discussion of complex clip.

Comparing complex numbers is challenging because the complex numbers aren't part of any ordered field. In NumPy, they're typically compared lexicographically: comparing the real part and only comparing the imaginary part if the real parts are equal. C++ and Python, on the other hand, do not support comparison ops on complex numbers.

Let's use this issue to enumerate complex comparison options as well as their pros and cons.

The current options are:

  • No default complex comparison
    • Pros:
      • Consistent with C++ and Python
      • Behavior is always clear since the user must specify the type of comparison
    • Cons:
      • Divergent from NumPy, but a clear error
      • Possibly inconvenient to always specify the comparison
  • Lexicographic comparison
    • Pros:
      • Consistent with NumPy
    • Cons:
      • Clamp (clip) behavior seems strange: (3 - 100j) clamped below to (2 + 5j) is unchanged, clamped above to (2 + 5j) becomes (2 + 5j)
      • Some users report wanting to compare complex by absolute value
  • Absolute value comparison
    • Pros:
      • Some applications naturally compare complex numbers using their absolute values
    • Cons:
      • Divergent from NumPy, possibly a silent break

cc. @rgommers @dylanbespalko @ezyang @mruberry
cc @ezyang @anjali411 @dylanbespalko

@anjali411 anjali411 added the module: complex Related to complex number support in PyTorch label Apr 12, 2020
@mruberry mruberry added the module: numpy Related to numpy support, and also numpy compatibility of our operators label Apr 12, 2020
@rgommers
Copy link
Collaborator

Behavior is always clear since the user must specify the type of comparison

this is the most compelling argument for me

@izdeby izdeby added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Apr 13, 2020
@dylanbespalko
Copy link
Contributor

dylanbespalko commented Apr 13, 2020

We need to develop kernels that

  1. Perform a intuitive function and
  2. Can benefit from fast hw acceleration.
  3. Are compatible with Numpy. I don't think anybody uses np.clip() for complex numbers.

@n-west had some really good ideas about making min/max different from clamp for complex numbers.

Clamp

  • Re-interpret the complex number as a float[2] and clamp the real/image components separately. This is not a comparison op because it produces two outputs (real/imag)
clamp(c, c_min, c_max) = complex(clamp(c.real, c_min.real, c_max.real),
                                 clamp(c.imag, c_min.imag, c_max.imag))
  • Intuitive: It performs the same clamp as a real number for both real and imag.
  • Fast: The fastest possible solution. Also faster than np.clip().

Min/Max

For Now: No default complex comparison. Nothing would currently benefit from hw acceleration.
In the Future: Absolute value comparison.

  • Intuitive: I think more people would vote for this comparison.
  • Fast: AVX512 has a hypot() command that accelerates abs() enabling the hw acceleration of min/max.

@gchanan
Copy link
Contributor

gchanan commented Apr 13, 2020

This problem seems similar to how nans are compared in floating point, i.e. there are some reasonable choices but no perfect definition.

@rgommers (or others who are familiar with NumPy) -- I wonder why NumPy decided to make the different choices here different functions as opposed to an optional argument, i.e. why define both amax and nanmax instead of amax(..., nans=ignore|max|min)?

@mruberry
Copy link
Collaborator

It seems like we have consensus on disabling complex comparison ops (and the ops which logically rely on them) for now.

@dylanbespalko Your idea for a complex clamp is interesting, but maybe we can find a new name for it?

@n-west
Copy link

n-west commented Apr 14, 2020

The proposed API from @dylanbespalko would work great for my applications! I like the idea of naming the complex clamp something else just to 100% prevent people from thinking np.clip and torch.clamp would do that same thing even though they are named differently (because for other datatypes they are the same operation AFAIK). No very clear operator name comes to mind though

@rgommers
Copy link
Collaborator

@rgommers (or others who are familiar with NumPy) -- I wonder why NumPy decided to make the different choices here different functions as opposed to an optional argument, i.e. why define both amax and nanmax instead of amax(..., nans=ignore|max|min)?

NumPy added nanmax and co in 2002 (numpy/numpy@cf5c05a45d571). At that point there really wasn't much thought about API design. I can't remember a discussion on changing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: complex Related to complex number support in PyTorch module: numpy Related to numpy support, and also numpy compatibility of our operators triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

No branches or pull requests

7 participants