Offset and scaling factors in axis format #4376 #6086

VincentVandalon · 2016-03-01T17:14:51Z

Feature/Ticket #4376 : Implemented functionality to let the user set an offset or
a scaling factor for a specific axis using ScalarFormatter.

Scaling and offset have become exclusive: either scaling or offset or nothing. This resolves ambiguity as (x-offset)/scalar != (x/scalar-offset)
Added a new method set_useScalingFactor(True|False|value)
Added a new method set_offset_string(string) that can override the automatic offset_string. This is useful when, for example, the used want to use a scalingFactor, but wants to display the scaling factor in the label: "Atomic density (10^14 atoms/nm^2)"
Wrote documentation for both set_useOffset(True|False|value) and set_useScalingFactor()
Updated existing documentation to match numpydoc format and updated documentation where I was sure of the purpose of the code
Fixed an (unreported) bug in HEAD that ignored the offset when scaling was performed automatically due to scientific notation (see discussion in ticket)
rcParams['axes.formatter.useoffset'] still controls whether to use offset when guessing best format.
set_powerlimits() still controls whether scaling is automatically performed

Possible code to test the features:

##########
#Y-axis tests default
#X-axis test user-set functions
#left column = scaling, right column = offset
#top row = (0...1)*1E10, bottom row = (0..1)+1E10
##########

import numpy as np
import scipy as sp
import matplotlib.pyplot as plt

scal=1E10

##############1
plt.subplot(221)
x=sp.rand(100)*scal
y=1000+x
plt.scatter(x,y)

plt.gca().get_xaxis().get_major_formatter().set_useScalingFactor(scal)


##############2
plt.subplot(222)
x=sp.rand(100)*scal
y=1000+x
plt.scatter(x,y)

plt.gca().get_xaxis().get_major_formatter().set_useOffset(scal)

##############3
plt.subplot(223)
x=sp.rand(100)+scal
y=1000+x
plt.scatter(x,y,c='r')

plt.gca().get_xaxis().get_major_formatter().set_useScalingFactor(scal)

##############4
plt.subplot(224)
x=sp.rand(100)+scal
y=1000+x
plt.scatter(x,y,c='r')

plt.gca().get_xaxis().get_major_formatter().set_useOffset(scal)
#plt.gca().get_xaxis().get_major_formatter().set_offset_string('')

plt.tight_layout()
plt.savefig('test.pdf')

a scaling factor for a specific axis using ScalarFormatter. - Scaling and offset have become exclusive: either scaling or offset or nothing. - This also resolves ambiguity as (x-offset)/scalar != (x/scalar-offset) - Added a new method set_useScalingFactor(True|False|value) - Wrote documentation for both set_useOffset(True|False|value) and set_useScalingFactor() - Fixed an (unreported) bug in HEAD that ignored the offset when scaling was performed automatically due to scientific notation - rcParams['axes.formatter.useoffset'] still controls whether to use offset when guessing best format. - set_powerlimits() still controls whether scaling is automatically performed

VincentVandalon · 2016-03-02T16:21:23Z

Does anybody have an idea what is causing the Latex errors (RuntimeError: LaTeX was not able to process the following string: '$$')?

I am not sure how this is related to the change I submitted (ticker.py is not in the stack trace). However, other PR's do not have this issue so it must be something in my code.

jenshnielsen · 2016-03-02T17:05:29Z

That is a random failure not related to anything in your PR, sorry about that. Did you intend to close the your pull request?

jenshnielsen · 2016-03-02T17:07:14Z

Sorry I was confused. We indeed have a number of random issues like that. But I don't think thats what happens here since it looks like it fails on all python versions

VincentVandalon · 2016-03-02T17:08:48Z

I closed the PR by accident. Nevertheless, I got the tests working locally, so I can fix some of the issues offline. When I have resolved the things I can fix myself, I will either make a new PR or ask for help. :)

QuLogic · 2016-03-02T20:05:01Z

No need to make a new PR; just push to the same branch,

VincentVandalon · 2016-03-05T23:16:12Z

I need some guidance: Due to a change in the code (for the better) some of the image tests/comparisons are failing. If an offset is used this is indicated at +1E10 and to make thing more symmetric scaling is now indicated with "·1E10" (note the cdot, previously none). This obviously causes the image tests to fail (see below).

Removing the cdot is an option, however, I think it really improves readability.

What is the next step? Should I change the test-images to include the cdot?

Completely off topic: what is a good toolset allowing one to switch between a devel version of matplotlib next to a normal version? I have been playing with virtualenv and will try conda next.

tacaswell · 2016-03-06T01:37:49Z

This is a case where replacing the test images makes sense.

I think a 'x' would make more sense than cdot. When I looked at the images before I read your text I thought the issue was that the minus sign was getting cut off!

I have used both venv and conda for flipping between mpl versions and either works. Conda in great on linux because there are binaries (when I was using venv, pip did not stash the bdist wheels it would build so it would have to reinstall from scratch. This was also way before I discovered ccache). conda also provides a version of qt/pyqt so you do not have to include system python packages, which is great if you want to flip numpy versions as well.

…y returned by get_useOffset(). get_useOffset() returns whether an offset has been set either by the user or by _set_offset()

…the axis

VincentVandalon · 2016-03-06T21:37:53Z

Thanks for the input! I changed the cdot to a cross, tacaswell's comment that the cdot could be mistaken for something else was something I did not consider.

I also changed the test images. To pass the test_axes.py test I re-implemented the option to have both an offset and a scaling factor. Why not support it, although it is not a normal use case (I hope, it hurts my brain).

Lets see what else I goofed up, my local test passed except for some unrelated errors with the svg and ps backend.

tacaswell · 2016-03-07T23:05:57Z

setup.cfg.template

@@ -13,7 +13,7 @@
 # set this to True.  It will download and build a specific version of
 # FreeType, and then use that to build the ft2font extension.  This
 # ensures that test images are exactly reproducible.
-#local_freetype = False


Please do not commit this change.

The same effect can be achived by

export MPLLOCALFREETYPE=1

in the enviroment you are building in.

Thanks for taking the time to review the changes this thoroughly! I am learning a lot from it.

Done (this change was not intended)

tacaswell · 2016-03-07T23:08:55Z

lib/matplotlib/ticker.py

+
+        Parameters
+        ----------
+        val : (True|False|numeric)


val : bool or scaler I think in the right numpydoc way to write this.

scalar not scaler.

tacaswell · 2016-03-14T01:41:14Z

lib/matplotlib/ticker.py

+
+        Parameters
+        ----------
+        s:  String describing the offset


There needs to be a space before the :

tacaswell · 2016-03-14T02:35:14Z

Sorry, I am having a lot of trouble following the changes here (hence my verbose questions).

I am not convinced that the logic of 'allowed to use an offset' and 'there is on offset worth using' can be merged into a single boolean because it should be responsive to scale changes. I am also not convinced that the handling of the rcparam is consistent with current behavior.

VincentVandalon · 2016-03-14T13:05:03Z

@tacaswell I gave your remark some thought. The logic existing before this fix was very opaque and scattered over multiple functions (good) in an illogical way (bad). For example, _set_offset should be named something along _calculateAutomaticOffset and should not contain logic at various positions to determine if that offset should be applied. That logic should be centralized somewhere else. Something similar holds for the very long _set_orderOfMagnitude(). So far I changed what was needed to get the desired functionality to work (see below). Maybe I should have started with a thorough cleanup.

I am willing to do the refactoring making the functions single purpose with the fewest possible side effects (as I have read up on the current code anyway), making the code in the ScalarFormatter more readable. On the other hand, this means that the change gets bigger and that will probably take more reviewing effort (although with better readable code). Before I start on this, I would like to know if there is support for such a step.

If we decide to refactor, you do not need to bother with the text from here on out.

Below I have previously written a clarification to the old/existing code and the new bits in reply to your post.

The code still uses the same/old functions for determining the scaling factor the offset implemented by [698] _set_orderOfMagnitude() and [667]_set_offset() respectively. Note that set_offset() is called at plot time and does not set the offset! I did not change any logic to determine the automatic behavior.

There are 3 booleans used to get the desired behavior (_usingScaling, _usingOffset, and _scientific) on top of the axes.formatter.useoffset and axes.formatter.limits. Moreover, orderOfMagnitude is used for the scaling value and offsetval is used for the offset value. As scientific notation is a specific case of a scaling factor, I used the majority of those functions. The only new parameter is _useScaling.

I have described the flow for several use cases to demonstrate that the rc-params are honored. Starting with the offset

No user input

[655] set_locs() is called at plot time -> this calls [667] _set_offset()
[667]_set_offset() evaluates the RC-param on [671].
If axes.formatter.useoffset == False the offset is not set, method return. Therefore the _usingOffset remains false
[606] get_offset() is called at plotting to add the label with the offset. This returns an empty string because _usingOffset is false.
If axes.formatter.useoffset == True the old algorithm to determine the offset is used.
[763] tick manipulation takes into account the offset

User input with set_useOffset(True) or set_useOffset(value)

[655] set_locs() is called at plot time -> this class [667] _set_offset() which returns because of if statement [671]. User has set the values
[606] get_offset() is called at plotting which returns the user set offset value
[763] tick manipulation takes into account the offset

User input with set_useOffset(False)

Identical to case 1), therefor honors axes.formatter.useoffset

For the scaling factor:

No user input

[655] set_locs() is called at plot time -> this calls [698] _set_orderOfMagnitude()
[698]_set_orderOfMagnitude() evaluates the RC-param self._powerlimits = rcParams['axes.formatter.limits']
on [719] and [721] and does what it has always doen
[606] get_offset() is called at plotting which returns a scaling factor determined with the already existing algorith. If the data is within axes.formatter.limits no scaling is applied.
[763] tick manipulation takes into account the scaling factor

User input set_useScalingFactor(True) or set_useScalingFactor(value)

[655] set_locs() is called at plot time -> this calls [698] _set_orderOfMagnitude()
[698]_set_offset() returns because of user input on line [672]
[606] get_offset() is called at plotting which returns the offset value set by the user
[763] tick manipulation takes into account the scaling factor

User input set_useScalingFactor(True)

Identical to case 1), therefor honors axes.formatter.limits

VincentVandalon · 2016-03-14T13:09:52Z

See above remark

- Refactored functionality of ScalarFormatter to be more readable - Separated logic deciding to perform auto scaling / offset from functions calculating the best possible scaling / offset values - Grouped methods in a logical way (user interaction, inherited, and local) - Removed space before the times sign in the scientific notation / scaling. Changed test images to reflect this. - Changed rcparam test back to original function - Removed _scientific boolean and the function set_scientific() as they were not used anywhere (just grep the original file). The existence of the function and boolean might mislead the user that changing this bool might affect the plotting.

VincentVandalon · 2016-03-17T22:25:48Z

lib/matplotlib/ticker.py

@@ -654,30 +626,76 @@ def get_offset(self):

    def set_locs(self, locs):


Start reading here to follow the flow of the code (this is called at plot time).

tacaswell · 2016-03-21T05:24:23Z

@anntzer Is also working on a major re-write of some of these code paths (#5804 and
#5785)

I am tentatively in favor of a major refactor. Given the importance of this code to many users day-to-day use I am sure it got written very early and has grown very organically over time. Which is to say, that it is a part of the code base that could most benefit from an overhaul

That said, this is also critical code path for most users day-to-day plotting which means even small API breaks can cause major disruption. Which means it is part of code base I am least excited about overhauling.

anntzer · 2016-03-21T05:32:12Z

I think #5785 (better choice of offset-text) is in pretty good shape and brings some helpful user-facing improvements; I'd like to see it (or some variant) merged. #5804 (complete rewrite of the formatter API) is also fine, but it's basically going to be very hard to get a decent rewrite of the mess that the formatter API is without breaking at least some obscure back-compatibilities, so I more or less gave up on it for now.

tacaswell · 2017-08-13T01:43:07Z

Closing as this has now been over a year without an update.

@VincentVandalon Thank you for you work and sorry this got stalled in review. If you are still interested in working on this please comment!

VincentVandalon added 5 commits March 1, 2016 13:18

Ticket 4376: Added more information in signature of the functions

ab1b3a2

Reformatted documentation of related function

4079e1a

Fixed minor issue, default scaling did not use scaling factor

bfc7389

Added functionality to manually set offset/scaling string

09f800c

mdboom added the status: needs review label Mar 1, 2016

tacaswell added this to the 2.1 (next point release) milestone Mar 2, 2016

VincentVandalon added 2 commits March 2, 2016 14:31

Made come more readable and more consistent with matplotlib defaults

02f0a11

Made code in ticker.py PEP8 compliant

5368336

VincentVandalon closed this Mar 2, 2016

mdboom removed the status: needs review label Mar 2, 2016

Removed old test and changed code to respect rcParams

fee8ab9

QuLogic reopened this Mar 2, 2016

QuLogic added the status: needs review label Mar 2, 2016

VincentVandalon added 4 commits March 6, 2016 14:58

Changes multiplication sign from cdot to cross

681820f

axis.formatter.useoffset is still honored, however, it is not directl…

66061ad

…y returned by get_useOffset(). get_useOffset() returns whether an offset has been set either by the user or by _set_offset()

Allowed both multiplication and scaling again

53ed618

Updated test images to handle addtion of x sign in scaling factor of …

4c86635

…the axis

PEP8 issues

9ce9d3c

tacaswell reviewed Mar 7, 2016
View reviewed changes

Minor improvements suggested by tacaswell

8d9d121

tacaswell reviewed Mar 14, 2016
View reviewed changes

lib/matplotlib/ticker.py

Parameters

----------

s: String describing the offset

Copy link

Member

tacaswell Mar 14, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There needs to be a space before the :

tacaswell added status: needs revision and removed status: needs review labels Mar 14, 2016

VincentVandalon reviewed Mar 17, 2016
View reviewed changes

tacaswell mentioned this pull request Sep 13, 2016

set offset threshold to 4 #7104

Closed

tacaswell closed this Aug 13, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Offset and scaling factors in axis format #4376 #6086

Offset and scaling factors in axis format #4376 #6086

VincentVandalon commented Mar 1, 2016

VincentVandalon commented Mar 2, 2016

jenshnielsen commented Mar 2, 2016

jenshnielsen commented Mar 2, 2016

VincentVandalon commented Mar 2, 2016

QuLogic commented Mar 2, 2016

VincentVandalon commented Mar 5, 2016

tacaswell commented Mar 6, 2016

VincentVandalon commented Mar 6, 2016

tacaswell Mar 7, 2016

VincentVandalon Mar 8, 2016

tacaswell Mar 7, 2016

QuLogic Mar 7, 2016

VincentVandalon Mar 8, 2016

tacaswell Mar 14, 2016

tacaswell commented Mar 14, 2016

VincentVandalon commented Mar 14, 2016

VincentVandalon commented Mar 14, 2016

VincentVandalon Mar 17, 2016

tacaswell commented Mar 21, 2016

anntzer commented Mar 21, 2016

tacaswell commented Aug 13, 2017

		@@ -654,30 +626,76 @@ def get_offset(self):

		def set_locs(self, locs):

Offset and scaling factors in axis format #4376 #6086

Offset and scaling factors in axis format #4376 #6086

Conversation

VincentVandalon commented Mar 1, 2016

VincentVandalon commented Mar 2, 2016

jenshnielsen commented Mar 2, 2016

jenshnielsen commented Mar 2, 2016

VincentVandalon commented Mar 2, 2016

QuLogic commented Mar 2, 2016

VincentVandalon commented Mar 5, 2016

tacaswell commented Mar 6, 2016

VincentVandalon commented Mar 6, 2016

tacaswell Mar 7, 2016

Choose a reason for hiding this comment

VincentVandalon Mar 8, 2016

Choose a reason for hiding this comment

tacaswell Mar 7, 2016

Choose a reason for hiding this comment

QuLogic Mar 7, 2016

Choose a reason for hiding this comment

VincentVandalon Mar 8, 2016

Choose a reason for hiding this comment

tacaswell Mar 14, 2016

Choose a reason for hiding this comment

tacaswell commented Mar 14, 2016

VincentVandalon commented Mar 14, 2016

VincentVandalon commented Mar 14, 2016

VincentVandalon Mar 17, 2016

Choose a reason for hiding this comment

tacaswell commented Mar 21, 2016

anntzer commented Mar 21, 2016

tacaswell commented Aug 13, 2017