trac user tillmann wrote on 2009-04-15

>>> import numpy as np
>>> x=np.array([[1,2],[3,4]]) 
>>> print x 
 [[1 2] 
 [3 4]] 
>>> x+=x.T 
>>> print x 
 [[2 5] 
 [8 8]]

@pv wrote on 2009-04-15

This was extensively discussed on the mailing list some time ago. IIRC, the outcome was that:

• there are valid use cases when it is important to allow this
• the behavior with views of multi-dimensional arrays is ill-defined, due to memory access pattern optimizations
• there's no way to determine whether an in-place operation is "valid" or not
• no decision was made on issuing a warning on in-place operation with a view on the LHS

This probably needs to be brought up again on the ML. Issuing a warning whenever an in-place operation involves a (N-dimensional?) view of the same array on the LHS is probably a sensible.

Milestone changed to 1.4.0 by @pv on 2009-04-15

Title changed from in place operation fail when used with transpose to Surprising results from in-place operations involving views by @pv on 2009-04-15

@pv wrote on 2009-04-15

More discussion here:

http://permalink.gmane.org/gmane.comp.python.numeric.general/29548

trac user lucaciti wrote on 2009-06-27

Hi!

I think tillmann is right. This behaviour is unexpected.
In my honest opinion, if numpy aims at competing with similar softwares, it has to protect the user from this kind of traps.

I think I fixed it, but please double-check whether it works as expected.
Basically: if an input argument and an output argument are one the view of the other (or are views of a common base array), further checks are performed. If they have exactly the same data pointer and the same strides, everything works as before. If not, the input argument is copied.

The following now works as expected:

In [1]: import numpy as N

In [2]: x = N.arange(10)

In [3]: y = x[::-1]

In [4]: y + x
Out[4]: array([9, 9, 9, 9, 9, 9, 9, 9, 9, 9])

In [5]: y += x

In [6]: x
Out[6]: array([9, 9, 9, 9, 9, 9, 9, 9, 9, 9])

In [7]: y
Out[7]: array([9, 9, 9, 9, 9, 9, 9, 9, 9, 9])

In [8]: x[0] = 6

In [9]: y
Out[9]: array([9, 9, 9, 9, 9, 9, 9, 9, 9, 6])

In [10]: x
Out[10]: array([6, 9, 9, 9, 9, 9, 9, 9, 9, 9])

In [11]: x = N.array([[1,2],[3,4]])

In [12]: x + x.T
Out[12]:
array([[2, 5],
       [5, 8]])

In [13]: x += x.T

In [14]: x
Out[14]:
array([[2, 5],
       [5, 8]])

In [15]: x = N.arange(20.).reshape(4,5)

In [16]: x / x[0]
Out[16]:
array([[         NaN,   1.        ,   1.        ,   1.        ,   1.        ],
       [         Inf,   6.        ,   3.5       ,   2.66666667,   2.25      ],
       [         Inf,  11.        ,   6.        ,   4.33333333,   3.5       ],
       [         Inf,  16.        ,   8.5       ,   6.        ,   4.75      ]])

In [17]: x /= x[0]

In [18]: x
Out[18]:
array([[         NaN,   1.        ,   1.        ,   1.        ,   1.        ],
       [         Inf,   6.        ,   3.5       ,   2.66666667,   2.25      ],
       [         Inf,  11.        ,   6.        ,   4.33333333,   3.5       ],
       [         Inf,  16.        ,   8.5       ,   6.        ,   4.75      ]])

while with the current implementation, the results for the in-place versions differ from the corresponding out-of-place versions.

Bye,

Luca

Attachment added by trac user lucaciti on 2009-06-27: inplace_views.patch

Attachment added by trac user lucaciti on 2009-06-27: inplace_views.2.patch

@pv wrote on 2009-06-28

Looks good! I don't think that anyone actually relies on in-place overwriting (as it's memory-order specific), even in 1D cases, but it might be good to recheck the old mailing list discussions if some good arguments were raised.

I think some work should be put in predicting "easy" special cases, in which copies can be avoided. For example, the following:

>>> x = np.arange(20000)
>>> x[:-1] += x[1:]

In general, solving this decision problem is probably hard, but most common special cases could probably be worked out.

Also, the prediction code should probably be put in a separate function, for clarity.

trac user lucaciti wrote on 2009-06-28

With the current implementation this works

>>> x = np.arange(20000)
>>> x[:-1] += x[1:]

but this doesn't:

>>> x = np.arange(20000)
>>> x[1:] += x[:-1]

With my patch both works but by making possibly unnecessary copies.
The quick answer would be to relax

1077        if (PyArray_DATA(obj) == PyArray_DATA(out) && PyArray_NDIM(obj) == PyArray_NDIM(out)) { ...

into

1077        if (PyArray_DATA(obj) >= PyArray_DATA(out) && PyArray_NDIM(obj) == PyArray_NDIM(out)) { ...

but I might be wrong.

trac user lucaciti wrote on 2009-06-28

Well, not, we should check for the sign of the strides...

@pv wrote on 2009-06-28

Also that is not enough, since larger strides can overtake smaller ones.

trac user lucaciti wrote on 2009-06-28

That is true.

I might be wrong but it might still work.
That line introduces a block checking whether all strides are the same. If they are the same, the two pointers proceed "parallel". If all strides are positive (which is a common case), the above check (>=) should suffice.

Of course, strides are very powerful but also very delicate and dangerous to handle. I might be completely wrong.

On the separate function thing, I definitely agree.

trac user lucaciti wrote on 2009-07-01

Hello!

I attach a new patch. The code which decides whether or not to make a copy is in a separate function.

If one input argument and one output argument are views of a same array, a copy is made unless:

• they have exactly the same strides and the same starting pointer (i.e. the two views overlap and are scanned in the same way);
• they have exactly the same strides, these are all positive, and the starting pointer of the output is lower than the input.

I also added some tests to check the correctness of the results.

Let me know if it works as expected.

Attachment added by trac user lucaciti on 2009-07-01: inplace_views_rev2.patch

@charris wrote on 2010-05-06

What are the current thoughts on this ticket? The patch does seem to deal with a common problem.

@pv wrote on 2010-05-06

@charris: based on the discussions on the ML, I think there is agreement that (i) we want this feature, (ii) provided it does not cause unnecessary copies in "simple" operations.

Regarding this patch, I suppose it only needs more eyes to verify that the approach does what is wanted.

Milestone changed to NumPy 2.0 by @pv on 2010-05-06

Attachment added by @charris on 2010-05-30: inplace_views_rev3.patch

@charris wrote on 2010-05-30

I'm looking at this and wondering if the ufuncs are the proper place for it. If we are only concerned with the extended assignment operators, then it would seem that that would be the proper place. The only normal use in a ufunc would be if output= was specified. I suppose the check could also be set up to take place in that circumstance.

I've also attached a patch that works for the current trunk.

@mwiebe wrote on 2011-03-24

A good approach for this might be to add a flag to the iterator like NPY_ITER_COPY_IF_OVERLAP. This would check if any READ or READWRITE operands overlap with READWRITE or WRITE operands, and make appropriate copies to avoid problems.

Just wondering, what is the status on this? I know it is as such ancient, but for take this (or well, something similar) might be interesting as well. I am thinking that it would make sense to make this, at least for starters, configurable with np.seterr (also avoiding speed issues with too many silenced warnings mostly)? Also I did not check it, but does the last comment mean you could cover this from the ufunc point of view mostly by doing it inside the interator?