Thanks to visit codestin.com
Credit goes to github.com

Skip to content

new image comparison unit tests #253

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
micahcochran opened this issue Jan 15, 2016 · 3 comments
Closed

new image comparison unit tests #253

micahcochran opened this issue Jan 15, 2016 · 3 comments

Comments

@micahcochran
Copy link
Contributor

Here are some ideas that I have about adding image comparison unit test. I want to get some input about this before I do too much coding on this.

  • Improve the examples/ to take CLI arguments that will plot to files when given --plot-file, and not display the plot to the screen when given --no-display. (--no-display is so that Travis will not get hung by any required interactivity.)
  • Create new image comparison unittests that call the examples/.
  • Since the examples/ are not installed, the unit tests should not be installed?

Or should the unit tests be installed? If they are installed it will add a few megabytes of test PNG images that will only be used when test() is called.

I am leaning towards not installed image comparison unit tests. Any thoughts? Is there something that I may not be considering?

I hope that this might lead to enough unittesting that could allow us to start the process of rewriting portions of the basemap code.
@micahcochran
Copy link
Contributor Author

I've started the above unit tests on my img-tests fork. This is not installed by setup.py.

The results seem pretty good. The test with matplotlib 1.4.3 is failing due to a high RMS between the images. (I've not seen how different the image looks.) "nightly" is failing because contourf has another instance of a numpy array being indexed by a casted floating point value due to division.

Before I increased the compare_image tolerance to a RMS of 10 all the test failed because the tolerance was a RMS of 0.001 RMS tolerance of 10 was determined by looking at what is typical for unit testing in matplotlib and geopandas.

@micahcochran
Copy link
Contributor Author

I installed matplotlib 1.4.3. At a glance, I can't really tell the difference between this and other generated versions. I've included images from that version. The img-tests fork is using a later version (1.5.x?).

Some options.

  1. I might be comfortable upping the RMS of these two tests a little.
  2. Another is just just allow the RMS to be higher for this particular version for these tests.
  3. A final option might be to drop support for this version and focus support 1.5.x for 2.0.x

Let me know. Thanks!

plotmap.png
plotmap
plotmap-failed-diff

simpletest.png
simpletest
simpletest-failed-diff

@WeatherGod
Copy link
Member

There are some lessons that can be learned from recent work in matplotlib:
matplotlib/matplotlib#5307

Not all of it can be taken advantage of right now, but some of it can be.
Another thing to consider is to ignore text for now in most of the tests,
focusing instead on correctness of the projections.

On Mon, Jan 25, 2016 at 5:10 PM, Micah Cochran [email protected]
wrote:

I installed matplotlib 1.4.3. At a glance, I can't really tell the
difference between this and other generated versions.

Some options.

  1. I might be comfortable upping the RMS of these two tests a little.
  2. Another is just just allow the RMS to be higher for this particular
    version for these tests.
  3. A final option might be to drop support for this version and focus
    support 1.5.x for 2.0.x

Let me know. Thanks!

plotmap.png
[image: plotmap]
https://cloud.githubusercontent.com/assets/7433104/12566040/362f3b50-c37d-11e5-9163-955f078d4753.png
[image: plotmap-failed-diff]
https://cloud.githubusercontent.com/assets/7433104/12566041/362f433e-c37d-11e5-822a-e38d3cde80c8.png

simpletest.png
[image: simpletest]
https://cloud.githubusercontent.com/assets/7433104/12566039/362d15b4-c37d-11e5-91a7-83636fefcf1f.png
[image: simpletest-failed-diff]
https://cloud.githubusercontent.com/assets/7433104/12566038/3622ecce-c37d-11e5-95d7-879e2f546bb9.png

—
Reply to this email directly or view it on GitHub
#253 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@WeatherGod @micahcochran