@taylor-scott very nice! I'm not super concerned with supporting NumPy 1.11, which is getting a bit long in the tooth, but the code is significantly cleaner now so I think this is a worthwhile change either way!

Regarding benchmarks, I think it's a good idea to add a couple (one for 2D and one for 3D), since performance is part of the motivation for this PR. I recommend you look at https://github.com/scikit-image/scikit-image/blob/8135c2f3505d77134162846abf1b06c0b18d5a56/benchmarks/benchmark_morphology.py for how to parameterise tests, and then add a class RegisterTranslation to benchmark_feature.py. As long as the class has a setup method and a time_register_translation method, it'll get picked up by airspeed velocity for benchmarks.

By the way, if this work sounds too onerous, we can do it the old-fashioned way: run a benchmark script on your machine and share with us the script and the results. =)

Honestly, I wouldn't worry about numpy 1.11 much. It is almost 3 years old now, so not so important.

I should write these out somewhere:

pip install asv

or

conda install asv

Then to test your benchmark

asv run -E existing -b RankSuite

To compare your results to master

asv continuous -b RankSuite -E conda:3.7 master simplify_benchmarks 

you might have to swap the order the branch names.

Hello @taylor-scott! Thanks for updating the PR.

• In the file benchmarks/benchmark_feature.py, following are the PEP8 issues :

Line 57:80: E501 line too long (88 > 79 characters)

Comment last updated on February 08, 2019 at 17:53 Hours UTC

Thanks for the help on benchmarking @jni and @hmaarrfk. I've pushed a benchmark file to this PR. It works with asv run -E existing but asv chokes while setting up new environments on my computer so I can't compare it to master.

I've run this this script several times and I've gotten pretty consistent results (Python 3.7, numpy 1.16.1). (Sorry it's a little long -- the easiest way to compare einsum and tensordot was to include the original and modified _upsampled_dft functions in the file.)

Results are here.

Highlights:

• 2D: np.tensordot is ~50 usec slower than np.dot
• 3D: np.tensordot is faster than np.einsum, particularly in the case of a larger image/upshift

Thoughts on using tensordot in the 2D case? It's slower, but the code cleaner with no special cases.

I think I'm nearly done with this...

A couple more things I noticed:

1. _upsampled_dft calculates the DFT but we actually want the inverse DFT. That's way the function call is _upsampled_df(image_product.conj(), ...).conj().

I think a good way to handle this is to add an optional flag inverse=False to _uspampled_dft. If true, use positive im2pi when calculating the kernel (to give the inverse DFT). If false, use -im2pi as it is now.

The conj calls can get expensive for large images (~10 msec for a 1040x1392) so I think it's good to get rid of them if possible.

2. L256: This might be returning the right thing by coincidence but max() sorts lexicographically for complex numbers. Looking at the matlab code, it's supposed to be the maximum cross_correlation sorted by absolute value. I'll fix this one tonight.

The MATLAB code is here:

CCabs = abs(CC);
[rloc, cloc] = find(CCabs == max(CCabs(:)),1,'first');
CCmax = CC(rloc,cloc);

It takes the absolute value of the cross-correlation, finds the argmax, then return the cross-correlation at that point.

The code in register_translation, CCmax = cross_correlation.max() returns the maximum value based on how numpy sorts complex numbers, which is not by absolute value. Instead, it sorts by real part first, then by complex part.

So:

np.array([1+1j, 0+10j]).max()

gives 1 + 1j even though abs(0+10j) > abs(1 + 1j)

See the discussion here: numpy/numpy#8151

@taylor-scott Sorry if my response was confusing; I was asking about item nr 1.

@stefanv I'm sorry, I completely misread your question.

To find the cross-correlation, we take the inverse DFT of the image product. In pixel-level registration, this is just cross_correlation = ifftn(image_product) but for subpixel registration the function calls _upsampled_dft which embeds the image product into a large image and calculates the DFT of the larger image.

This almost gives the cross-correlation, but we actually need the inverse DFT of the larger image. So to calculate the cross-correlation, register_translation uses the identity fft(image.conj).conj/N = ifft(image) where N is the image size.

For large images, the time to call conj is not trivial. For example, calling image.conj() on one of my images (1040 x 1392) takes ~11 msec and registering two images takes ~230 msec total.

To avoid this call to conj, I proposed adding an optional flag inverse=False to _upsampled_dft. When false, return the DFT as the function does now. When true, return the inverse DFT which is what we want in this case.

This is easy to accomplish while generating the kernel:

    kernel = ((np.arange(upsampled_region_size[0]) - ax_offset)[:, None]
              * np.fft.fftfreq(n_items, upsample_factor))
    if inverse:
        # We can also add the normalizing factor here if desired
        kernel = np.exp(im2pi * kernel)
    else:
        kernel = np.exp(-im2pi * kernel)

Also, after thinking about it a bit more, this may be an issue better handled in a separate pull request after some more discussion. I think _upsampled_dft could be generally useful, but IMO this aspect and how the image is embedded in the upsampled image are non-intuitive.

@taylor-scott Thank you for that very clear explanation. Yikes, such subtle tricks are very easy to miss, and I agree with you that making it explicit would be an improvement. If you are willing to make the change in another PR, then great—let's get this one merged so long. Feel free to add your name to CONTRIBUTORS.txt if you want.

@scikit-image/core Ready for final review.

Ok, great. I've removed the inverse flag from this PR and I'll open a separate PR to address that. Thanks @stefanv !

Thanks, @taylor-scott!