Comparison ops are not implemented for complex numbers, this is expected behavior #36444 (comment)

In my opinion, this operation makes sense, and it could potentially be performed independently on the real and imaginary part independently.

clamp is an operation that's used for stability to clamp values that are too small or too large. Too small or too large can be understood in a metric sense (complex numbers do have a metric) as "I don't wan't to have any value that has norm larger/smaller than X". The proposal above uses a variation of this idea via the norm 1.

I wanted to use this operation for the backward of the LU decomposition, and in the end we had to work around it by implementing this operation manually.

While the general rule of "we don't support comparison operations for complex numbers" is indeed the correct one, we can consider implementing some operations that are not so "mathematical" in appropriate ways, as long as we document this behaviour accordingly.

@anjali411 wdyt?

Does numpy implement clamp in this way? :)

Here is the problem, I got some data whose type is complex, and I want to train a network on these data, as I wrote

there is a runtime error when I run this code, I don't know why it will call clamp_min function in the forward process？

it seems that numpy doesn't implement clamp for complexdouble

What semantics do you expect from relu operation on complex inputs?

What semantics do you expect from relu operation on complex inputs?

Oh, I know, but does PyTorch has any non-linear active function for complex?

Any non-linear activation that does not depend on min/max certainly works for complex numbers (i.e. pretty much all of them but those with cut-offs and those that are variations of relu).

How do you do a workaround of this issue? splitting into real and imaginary?

Clamping the real and imaginary parts of a complex tensor separately is possible; one option for "clamping" complex numbers is to compute a complex number with the same angle/phase but scaled magnitude. In this case, however, a hypothetical clamp_magnitude would accept a minimum and maximum magnitude.

@SantaTitular if you want to clamp the absolute value, then this is probably the recommended way.

a = torch.randn(4, dtype=torch.cfloat)
angle_ = a.angle()
clamped_abs = torch.clamp(a.abs(), min=...)
b = torch.polar(clamped_abs, angle_)

If you want to clamp the real or imag value for some reason, then that can easily done by accessing the real and imag attributes on the tensor (they are views!) and clamping them.

Thank you for the fast response @mruberry @anjali411 .

Indeed both your ideas would work nicely (I'll use it as a reference for future applications). For my case, the application was using a complex ReLu DEEP COMPLEX NETWORKS
, similar to @Fun772283153 . I was testing a simple MLP (below) and I think it is a nice workaround, just have to get the Loss function (BCE, CrossEntropyLoss) to accept complex numbers to test it out (@mruberry not sure if you know something about it 😅, I already looked into #46642)

class Feedforward(torch.nn.Module):
        def __init__(self, input_size, hidden_size):
            super(Feedforward, self).__init__()
            self.input_size = input_size
            self.hidden_size  = hidden_size
            self.fc1 = torch.nn.Linear(self.input_size, self.hidden_size).to(torch.complex128)
            self.relu = torch.nn.ReLU()
            self.fc2 = torch.nn.Linear(self.hidden_size, 1).to(torch.complex128)
            self.sigmoid = torch.nn.Sigmoid()
        def forward(self, x):
            hidden = self.fc1(x)
            relu = self.relu(hidden.real) + 1j * self.relu(hidden.imag)
            output = self.fc2(relu)
            output = self.sigmoid(output)
            return output

@anjali411 is definitely our expert on complex losses -- in fact she's preparing a blog post that includes a complex neural network now!

Damn, thats great news! I'm very interested in reading and commenting on it. @anjali411 please let me know when it is available (not sure if there is a way on github to track/read about new blog posts)