Boxplot fix median line extending past box boundaries #19409 #26462

kidkoder432 · 2023-08-07T03:09:03Z

PR summary

Fixes #19409
The median line on a boxplot would extend past the box when the line width was increased. This PR aims to fix the visual annoyance by setting the solid line capstyle to be the same as all other capstyles. additionally, this change keeps the capstyle of the median lines consistent with other boxplot lines (i.e. mean lines).

This is my second ever PR so be nice please :)

Other changes:

Fixed a small typo in bxp() docstring
Added a new test for the median line (from PR Boxplot: Median line is now bound within the box for all linewidths #23335)

PR checklist

[N/A] "closes #0000" is in the body of the PR description to link the related issue
new and changed code is tested
[N/A] Plotting related features are demonstrated in an example
[N/A] New Features and API Changes are noted with a directive and release note
Documentation complies with general and docstring guidelines

Other changes: - Update `.gitignore` to ignore `venv/` in commit - Fixed a small typo in `bxp()` - Added a new test for the median line (from PR matplotlib#23335)

github-actions

Thank you for opening your first PR into Matplotlib!

If you have not heard from us in a week or so, please leave a new comment below and that should bring it to our attention. Most of our reviewers are volunteers and sometimes things fall through the cracks.

You can also join us on gitter for real-time discussion.

For details on testing, writing docs, and our review process, please see the developer guide

We strive to be a welcoming and open project. Please follow our Code of Conduct.

melissawm · 2023-08-07T19:50:18Z

Hi @kidkoder432 - while we wait for a proper review, would you mind taking a look at the Linting failure here? https://github.com/matplotlib/matplotlib/actions/runs/5780696475/job/15664707497?pr=26462

It looks like you are missing a blank line in your test.

rcomer

Thank you for your work on this @kidkoder432, but I do not think this is currently the right change.

lib/matplotlib/axes/_axes.py

lib/matplotlib/lines.py

kidkoder432 · 2023-08-08T23:06:15Z

Oh okay, thanks! (This is my first contribution to matplotlib so I appreciate the feedback :))

…

On Tue, Aug 8, 2023 at 12:01 AM Ruth Comer ***@***.***> wrote: ***@***.**** commented on this pull request. Thank you for your work on this @kidkoder432 <https://github.com/kidkoder432>, but I do not think this is currently the right change. ------------------------------ In lib/matplotlib/axes/_axes.py <#26462 (comment)> : > @@ -4095,7 +4095,7 @@ def bxp(self, bxpstats, positions=None, widths=None, vert=True, capwidths : float or array-like, default: None Either a scalar or a vector and sets the width of each cap. - The default is ``0.5*(with of the box)``, see *widths*. + The default is ``0.5*(width of the box)``, see *widths*. Nice spot! ------------------------------ In lib/matplotlib/lines.py <#26462 (comment)> : > @@ -297,7 +297,7 @@ def __init__(self, xdata, ydata, *, markerfacecoloralt='none', fillstyle=None, antialiased=None, - dash_capstyle=None, + dash_capstyle="butt", I think you intended to change *solid_capstyle* rather than *dash_capstyle*, but I do not think we want to do this here. This will define the default capstyle for lines used all over the library, and will override what the user has set in their rcParams (see the logic below with if solid_capstyle is None:, etc.) For the current issue, I think we only want to change the capstyle used by default within boxplot for the median line. As noted in #19409 <#19409>, you can get at that in the medianprops dictionary. — Reply to this email directly, view it on GitHub <#26462 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOYIALWGZ6KQXTFQWGJIMH3XUHP5BANCNFSM6AAAAAA3GLM3QA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

rcomer · 2023-08-11T12:34:27Z

This change has caused some test failures. @kidkoder432 are you confident to investigate those yourself? You can see the full output from the tests by clicking the "Details" link by one of the checks at the bottom of this page. You can also run the tests yourself locally:
https://matplotlib.org/devdocs/devel/testing.html

kidkoder432 · 2023-08-12T00:51:56Z

This change has caused some test failures. @kidkoder432 are you confident to investigate those yourself? You can see the full output from the tests by clicking the "Details" link by one of the checks at the bottom of this page. You can also run the tests yourself locally: https://matplotlib.org/devdocs/devel/testing.html

Sure, I'll take a look and let you know any concerns.

kidkoder432 · 2023-08-12T01:12:20Z

Edit: I fixed the NoneType errors that were occurring when a medianprops dictionary was not provided, but those same tests now fail with ImageComparisonFailure:

Okay, so I looked at the failing tests and noticed that the expected images were all using a capstyle of projecting. However, because I modified the code to use a capstyle of butt. the original tests for median lines were failing because they were expecting a projecting capstyle. However, this capstyle is the reason why the median line was extending past the box. According to the docs:

'projecting'
the line is squared off as in butt, but the filled-in area extends beyond the endpoint a distance of linewidth/2.

I'm not sure how to move forward with this.

Sometimes the `medianprops` dict was not provided so the previous code was failing to assign a key to it. So a check was added to either modify an exsisting dict or create a new one.

rcomer · 2023-08-12T07:46:33Z

OK, if the user specifies a capstyle then we should assume they are happy with their choice and leave it. We should only set it to “butt” if the user didn’t make a choice. I think the dictionary setdefault method might be useful here.

kidkoder432 · 2023-08-13T01:54:45Z

OK, if the user specifies a capstyle then we should assume they are happy with their choice and leave it. We should only set it to “butt” if the user didn’t make a choice. I think the dictionary setdefault method might be useful here.

Good idea.
Those tests don't seem to define a capstyle specifically though, so they still fail...

rcomer · 2023-08-14T18:29:50Z

Ah OK, I thought you meant that some tests were specifying “projecting” as the capstyle. Instead they are just leaving it to default and this change switches the default from “projecting” to “butt”. So the tests are failing because of a genuine (desired) change in the output, which means you will need to replace the reference images. We have the triage_tests tool to help with that.

I used the triage tests tool to set the test benchmark images to those with a capstyle of `butt`. I also modieifed the behaviror for setting the capstyle for median lines to obey the user's preference instead of overriding it.

rcomer · 2023-08-16T19:00:46Z

Thanks for the update @kidkoder432. It looks like you still have a couple of image test failures for pdf files. Do you see those failures locally? If not you might be missing a dependency - most likely ghostscript.
https://matplotlib.org/devdocs/devel/dependencies.html#dependencies-for-testing-matplotlib

kidkoder432 · 2023-08-16T19:08:42Z

i don't see them locally; i probably do need to install ghostscript. How would I resolve the PDF test failures?

…

On Wed, Aug 16, 2023, 12:00 PM Ruth Comer ***@***.***> wrote: Thanks for the update @kidkoder432 <https://github.com/kidkoder432>. It looks like you still have a couple of image test failures for pdf files. Do you see those failures locally? If not you might be missing a dependency - most likely ghostscript. https://matplotlib.org/devdocs/devel/dependencies.html#dependencies-for-testing-matplotlib — Reply to this email directly, view it on GitHub <#26462 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOYIALX7ALTX6FGE4YKIQ33XVUKGTANCNFSM6AAAAAA3GLM3QA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

rcomer · 2023-08-16T19:10:22Z

I think the triage_tests script should handle pdf as well as png.

kidkoder432 · 2023-08-16T19:21:49Z

Got it, thank you!

…

On Wed, Aug 16, 2023, 12:10 PM Ruth Comer ***@***.***> wrote: I think the triage_tests script should handle pdf as well as png. — Reply to this email directly, view it on GitHub <#26462 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOYIALQJRPPJJSWYAPEUTYLXVULKTANCNFSM6AAAAAA3GLM3QA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

kidkoder432 · 2023-08-18T03:00:14Z

I'm having trouble getting pytest to recognize that I have Ghostscript installed. are there any resources for this on the matplotlib docs?

i used the installer to install Ghostscript 10.01.2.

ksunden · 2023-08-18T07:17:00Z

Matplotlib only interacts with ghostscript via subprocess calls. Can you call gs as a CLI program (e.g. gs --help) (on windows it may also be called gswin64c, not really sure what is fully going on there...)

If it is not in your PATH, it will not be found, so make sure it is installed in a place that it can be executed.

kidkoder432 · 2023-08-19T01:22:55Z

I use VSCode for development. If I run gswin64 from a regular powershell terminal, it launches a command window to type gs commands with (gs is in system PATH). However, this command does not work in VSCode, and pdf tests are skipped by pytest. How do I resolve this so i can continue development? Thanks.

…

On Fri, Aug 18, 2023 at 12:17 AM Kyle Sunden ***@***.***> wrote: Matplotlib only interacts with ghostscript via subprocess calls. Can you call gs as a CLI program (e.g. gs --help) (on windows it may also be called gswin64c, not really sure what is fully going on there...) If it is not in your PATH, it will not be found, so make sure it is installed in a place that it can be executed. — Reply to this email directly, view it on GitHub <#26462 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOYIALUH646NVJQ3JGQTYZLXV4JHRANCNFSM6AAAAAA3GLM3QA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

kidkoder432 · 2023-08-19T01:25:05Z

Never mind, i got it to work.

…

On Fri, Aug 18, 2023, 6:22 PM prajwal agrawal ***@***.***> wrote: I use VSCode for development. If I run gswin64 from a regular powershell terminal, it launches a command window to type gs commands with (gs is in system PATH). However, this command does not work in VSCode, and pdf tests are skipped by pytest. How do I resolve this so i can continue development? Thanks. On Fri, Aug 18, 2023 at 12:17 AM Kyle Sunden ***@***.***> wrote: > Matplotlib only interacts with ghostscript via subprocess calls. Can you > call gs as a CLI program (e.g. gs --help) (on windows it may also be > called gswin64c, not really sure what is fully going on there...) > > If it is not in your PATH, it will not be found, so make sure it is > installed in a place that it can be executed. > > — > Reply to this email directly, view it on GitHub > <#26462 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AOYIALUH646NVJQ3JGQTYZLXV4JHRANCNFSM6AAAAAA3GLM3QA> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

The boxplot PDFs were using the wrong capstyle, so I modified the references to use the butt captyle.

kidkoder432 · 2023-08-19T04:48:09Z

Hi, for some reason it says that CodeCov tests failed in the workflow logs. However, when I visit the website, its says that codecov tests all passed. Can someone please take a look at this?

lib/matplotlib/axes/_axes.py

rcomer · 2023-08-19T09:13:12Z

I wouldn’t worry about that codecov failure. We currently use codecov for information only. In this case, codecov says that 100% of your change is covered, which I think is the important thing. The coverage reductions are all “indirect changes” - I haven’t really got my head around how those are caused myself.

.gitignore

Co-authored-by: Elliott Sales de Andrade <[email protected]>

kidkoder432 · 2023-08-19T17:41:43Z

Yes, it makes sense. Thank you for the explanation!

…

On Sat, Aug 19, 2023 at 10:40 AM Ruth Comer ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In lib/matplotlib/axes/_axes.py <#26462 (comment)> : > + if medianprops is None: # Clamp median line to edge of box + medianprops = {"solid_capstyle": "butt"} + else: + medianprops.setdefault("solid_capstyle", "butt") Good question. Suppose a user has a dictionary of properties that they re-use for multiple different plot types in their script. If that dictionary gets added to when they pass it to boxplot, then it could affect other plots that are created further down the script. That would be unexpected (and therefore undesirable) behaviour. Does that make sense? You are right that setdefault won’t change the dictionary if it already has that key. — Reply to this email directly, view it on GitHub <#26462 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOYIALRSMA26DNRGEXL4GNDXWD3CHANCNFSM6AAAAAA3GLM3QA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

rcomer

This looks good to me now. For code changes, we require approvals from two reviewers so this can’t be merged just yet. Thank you for your work on it @kidkoder432!

timhoffm

Question: Does anybody understand why the box borders are sometimes shifted slightly?

Comment: The original issue also discussed bringing the line below the box using zorder. This is not discussed in this PR, While conceptually desireable, this is not so easy to fix: The box can be filled and then face and edge are drawn with the same zorder. This means, one cannot draw mean/median lines below the box edge but above the box face (at least not without complicating the draw logic).
I'm therefore inclinded to accept the PR without addressing zorder, but would like to have at least one second thought on this.

timhoffm · 2023-08-20T21:07:15Z

lib/matplotlib/axes/_axes.py

@@ -4147,6 +4147,11 @@ def bxp(self, bxpstats, positions=None, widths=None, vert=True,
        --------
        .. plot:: gallery/statistics/bxp.py
        """
+        # Clamp median line to edge of box by default.
+        medianprops = {
+            "solid_capstyle": "butt",


I think we should unversally "solid_capstyle" and "dashed_capstyle" to "butt" for both medianprops and meanprops. We want the lines to end within the box for both independent of the global capstyle settings.

kidkoder432 · 2023-08-20T23:55:19Z

Question: Does anybody understand why the box borders are sometimes shifted slightly?

Comment: The original issue also discussed bringing the line below the box using zorder. This is not discussed in this PR, While conceptually desireable, this is not so easy to fix: The box can be filled and then face and edge are drawn with the same zorder. This means, one cannot draw mean/median lines below the box edge but above the box face (at least not without complicating the draw logic).
I'm therefore inclinded to accept the PR without addressing zorder, but would like to have at least one second thought on this.

Not sure why either... From what you explained about the draw logic it would likely be much easier to stay with the current implementation.

rcomer · 2023-08-21T06:58:14Z

Question: Does anybody understand why the box borders are sometimes shifted slightly?

Do you have an example? When I looked at the test images that have changed here, all the diffs seemed to just show the median line getting shorter.

I'm therefore inclinded to accept the PR without addressing zorder, but would like to have at least one second thought on this.

I came to same conclusion based on @oscargus' comment on the previous PR: #23335 (comment)

timhoffm · 2023-08-21T20:15:10Z

Do you have an example? When I looked at the test images that have changed here, all the diffs seemed to just show the median line getting shorter.

Can't find anymore. Either I was wrong or this was somehow fixed through the recent updates.

timhoffm · 2023-08-21T20:21:24Z

Since this is a bugfix, I recon that it can still go into 3.8.0 (on the basis that we do not have release candidates for any bugfix releases, there's no difference between deferring to 3.8.1 and pushing this into 3.8.0 even after 3.8.0rc1).

…box boundaries matplotlib#19409

timhoffm · 2023-08-21T20:28:21Z

Thanks @kidkoder432, and congratulations on your first contrbution to matplotlib! We'd happy to see you back!

…462-on-v3.8.x Backport PR #26462 on branch v3.8.x (Boxplot fix median line extending past box boundaries #19409)

kidkoder432 · 2023-08-22T01:13:03Z

You're welcome! Thank you for your support! I'll definitely be back to contribute :)

…

On Mon, Aug 21, 2023, 1:28 PM Tim Hoffmann ***@***.***> wrote: Thanks @kidkoder432 <https://github.com/kidkoder432>, and congratulations on your first contrbution to matplotlib! We'd happy to see you back! — Reply to this email directly, view it on GitHub <#26462 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOYIALXG77XIXJIN4LCEWILXWPAHDANCNFSM6AAAAAA3GLM3QA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

kidkoder432 added 2 commits August 6, 2023 19:52

Set default capstyle to butt

53f5132

Other changes: - Update `.gitignore` to ignore `venv/` in commit - Fixed a small typo in `bxp()` - Added a new test for the median line (from PR matplotlib#23335)

fix typo

5d0a323

github-actions bot reviewed Aug 7, 2023

View reviewed changes

Remove venv/ in ignore list

67d26cc

rcomer reviewed Aug 8, 2023

View reviewed changes

lib/matplotlib/axes/_axes.py Show resolved Hide resolved

lib/matplotlib/lines.py Outdated Show resolved Hide resolved

rcomer added status: needs revision PR: bugfix Pull requests that fix identified bugs labels Aug 8, 2023

fix formatting & remove capstyle in default params

6571818

kidkoder432 added 2 commits August 11, 2023 18:16

fix NoneType error test failure

b1d8b51

Sometimes the `medianprops` dict was not provided so the previous code was failing to assign a key to it. So a check was added to either modify an exsisting dict or create a new one.

fix formatting

0a9d0f8

Fixed tests to use butt capstyle

d1ba031

I used the triage tests tool to set the test benchmark images to those with a capstyle of `butt`. I also modieifed the behaviror for setting the capstyle for median lines to obey the user's preference instead of overriding it.

fixed pdf test failures

f2fdc2c

The boxplot PDFs were using the wrong capstyle, so I modified the references to use the butt captyle.

QuLogic reviewed Aug 19, 2023

View reviewed changes

lib/matplotlib/axes/_axes.py Outdated Show resolved Hide resolved

rcomer removed the status: needs revision label Aug 19, 2023

rcomer reviewed Aug 19, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

rcomer mentioned this pull request Aug 19, 2023

Boxplot: Median line is now bound within the box for all linewidths #23335

Closed

3 tasks

kidkoder432 and others added 2 commits August 19, 2023 08:38

Update .gitignore

8536a35

Update _axes.py

0f2dee1

Co-authored-by: Elliott Sales de Andrade <[email protected]>

rcomer approved these changes Aug 19, 2023

View reviewed changes

timhoffm reviewed Aug 20, 2023

View reviewed changes

Force dash_capstyle to butt (unless specified)

0e98cd8

timhoffm approved these changes Aug 21, 2023

View reviewed changes

timhoffm added this to the v3.8.0 milestone Aug 21, 2023

timhoffm merged commit 813e41a into matplotlib:main Aug 21, 2023

meeseeksmachine mentioned this pull request Aug 21, 2023

Backport PR #26462 on branch v3.8.x (Boxplot fix median line extending past box boundaries #19409) #26568

Merged

meeseeksmachine pushed a commit to meeseeksmachine/matplotlib that referenced this pull request Aug 21, 2023

Backport PR matplotlib#26462: Boxplot fix median line extending past …

4ffe411

…box boundaries matplotlib#19409

ksunden added a commit that referenced this pull request Aug 21, 2023

Merge pull request #26568 from meeseeksmachine/auto-backport-of-pr-26…

017f729

…462-on-v3.8.x Backport PR #26462 on branch v3.8.x (Boxplot fix median line extending past box boundaries #19409)

ksunden mentioned this pull request Sep 15, 2023

merge up v3.8.0 #26785

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Boxplot fix median line extending past box boundaries #19409 #26462

Boxplot fix median line extending past box boundaries #19409 #26462

kidkoder432 commented Aug 7, 2023 •

edited

Loading

github-actions bot left a comment

melissawm commented Aug 7, 2023

rcomer left a comment

kidkoder432 commented Aug 8, 2023 via email

rcomer commented Aug 11, 2023

kidkoder432 commented Aug 12, 2023

kidkoder432 commented Aug 12, 2023 •

edited

Loading

rcomer commented Aug 12, 2023 •

edited

Loading

kidkoder432 commented Aug 13, 2023

rcomer commented Aug 14, 2023

rcomer commented Aug 16, 2023

kidkoder432 commented Aug 16, 2023 via email

rcomer commented Aug 16, 2023

kidkoder432 commented Aug 16, 2023 via email

kidkoder432 commented Aug 18, 2023

ksunden commented Aug 18, 2023

kidkoder432 commented Aug 19, 2023 via email

kidkoder432 commented Aug 19, 2023 via email

kidkoder432 commented Aug 19, 2023

rcomer commented Aug 19, 2023

kidkoder432 commented Aug 19, 2023 via email

rcomer left a comment

timhoffm left a comment

timhoffm Aug 20, 2023

kidkoder432 Aug 20, 2023 •

edited

Loading

kidkoder432 commented Aug 20, 2023

rcomer commented Aug 21, 2023

timhoffm commented Aug 21, 2023

timhoffm commented Aug 21, 2023

timhoffm commented Aug 21, 2023

kidkoder432 commented Aug 22, 2023 via email

Boxplot fix median line extending past box boundaries #19409 #26462

Boxplot fix median line extending past box boundaries #19409 #26462

Conversation

kidkoder432 commented Aug 7, 2023 • edited Loading

PR summary

PR checklist

github-actions bot left a comment

Choose a reason for hiding this comment

melissawm commented Aug 7, 2023

rcomer left a comment

Choose a reason for hiding this comment

kidkoder432 commented Aug 8, 2023 via email

rcomer commented Aug 11, 2023

kidkoder432 commented Aug 12, 2023

kidkoder432 commented Aug 12, 2023 • edited Loading

rcomer commented Aug 12, 2023 • edited Loading

kidkoder432 commented Aug 13, 2023

rcomer commented Aug 14, 2023

rcomer commented Aug 16, 2023

kidkoder432 commented Aug 16, 2023 via email

rcomer commented Aug 16, 2023

kidkoder432 commented Aug 16, 2023 via email

kidkoder432 commented Aug 18, 2023

ksunden commented Aug 18, 2023

kidkoder432 commented Aug 19, 2023 via email

kidkoder432 commented Aug 19, 2023 via email

kidkoder432 commented Aug 19, 2023

rcomer commented Aug 19, 2023

kidkoder432 commented Aug 19, 2023 via email

rcomer left a comment

Choose a reason for hiding this comment

timhoffm left a comment

Choose a reason for hiding this comment

timhoffm Aug 20, 2023

Choose a reason for hiding this comment

kidkoder432 Aug 20, 2023 • edited Loading

Choose a reason for hiding this comment

kidkoder432 commented Aug 20, 2023

rcomer commented Aug 21, 2023

timhoffm commented Aug 21, 2023

timhoffm commented Aug 21, 2023

timhoffm commented Aug 21, 2023

kidkoder432 commented Aug 22, 2023 via email

kidkoder432 commented Aug 7, 2023 •

edited

Loading

kidkoder432 commented Aug 12, 2023 •

edited

Loading

rcomer commented Aug 12, 2023 •

edited

Loading

kidkoder432 Aug 20, 2023 •

edited

Loading