Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Bug]: matplotlib.pyplot.clf is very slow #23771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
eendebakpt opened this issue Aug 29, 2022 · 8 comments
Open

[Bug]: matplotlib.pyplot.clf is very slow #23771

eendebakpt opened this issue Aug 29, 2022 · 8 comments
Labels
Performance

Comments

@eendebakpt
Copy link
Contributor

Bug summary

For relatively simple plotting examples the plt.clf() command can take over 30% of the calculation time. One would expect that a clear command is relatively simple and takes a limited amount of time

Code for reproduction

import numpy as np
import time
import matplotlib
import matplotlib.pyplot as plt
print(matplotlib.__version__)

def profile_expression(expression):
    import cProfile  
    import tempfile
    import os
    import subprocess
    tmpdir = tempfile.mkdtemp()
    statsfile = os.path.join(tmpdir, "profile_expression_stats")
    print("profile_expression: running")
    t0 = time.perf_counter()
    cProfile.run(expression, statsfile)
    dt = time.perf_counter() - t0
    print(f"profiling: {dt:.2f} [s]")
    r = subprocess.Popen(["snakeviz", statsfile])
    return r

x=np.linspace(0, 10, 1000)
y=np.linspace(0, 20, 1000)**0.1

def go():
    for ii in range(5):
        plt.figure(100); 
        plt.clf()
        plt.plot(x, y, '.', label='n')
        plt.plot(x, y+x, '.-', label='m')
        plt.xlabel('dsf')
        plt.title('sdfs')
        plt.legend()
        plt.draw()

if __name__=='__main__':
    profile_expression('go()')

Actual outcome

The output of the minimal example (which requires https://github.com/jiffyclub/snakeviz to be installed) is a profiling graph of the code executed in the minimal example:

image

Expected outcome

The plt.clf() should be very fast compared to the actual plotting. Going into the details of the profiling results shows a large amount of time being spend in axis.reset_ticks.

Additional information

No response

Operating system

Windows

Matplotlib Version

3.6.0.dev3058+g87197156eb

Matplotlib Backend

Qt5Agg

Python version

3.10.0

Jupyter version

No response

Installation

git checkout
@timhoffm
Copy link
Member

timhoffm commented Aug 29, 2022 •
edited
Loading

Tick handling is a known performance issue, but it's hard to change due to compatibility (see #5665 for a general discussion).

We already have some optimizations, but they may not apply to reset_ticks(). How is the timing when you discard the figure and create a new one instead? It might be that this is faster:

    for ii in range(5):
        plt.figure(ii); 
        plt.plot(x, y, '.', label='n')
        plt.plot(x, y+x, '.-', label='m')
        plt.xlabel('dsf')
        plt.title('sdfs')
        plt.legend()
        plt.draw()
        plt.close()

Edit: The clearing issue is also described/addressed in #16067

@tacaswell
Copy link
Member

I'm also a bit surprised how much time we are spending in the deprecation machinery....

@eendebakpt
Copy link
Contributor Author

I'm also a bit surprised how much time we are spending in the deprecation machinery....

The overhead of the deprecation is really low. The reported times in the profiling graph are cummulative which is why they show up. What you see is just this:

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        # Don't use signature.bind here, as it would fail when stacked with
        # rename_parameter and an "old" argument name is passed in
        # (signature.bind would fail, but the actual call would succeed).
        if len(args) > name_idx:
            warn_deprecated(
                since, message="Passing the %(name)s %(obj_type)s "
                "positionally is deprecated since Matplotlib %(since)s; the "
                "parameter will become keyword-only %(removal)s.",
                name=name, obj_type=f"parameter of {func.__name__}()")
        return func(*args, **kwargs)

@eendebakpt
Copy link
Contributor Author

eendebakpt commented Aug 30, 2022 •
edited by timhoffm
Loading

Tick handling is a known performance issue, but it's hard to change due to compatibility (see #5665 for a general discussion).

We already have some optimizations, but they may not apply to reset_ticks(). How is the timing when you discard the figure and create a new one instead? It might be that this is faster:

    for ii in range(5):
        plt.figure(ii); 
        plt.plot(x, y, '.', label='n')
        plt.plot(x, y+x, '.-', label='m')
        plt.xlabel('dsf')
        plt.title('sdfs')
        plt.legend()
        plt.draw()
        plt.close()

Edit: The clearing issue is also described/addressed in #16067

Creating new figures (e.g. plt.figure(ii) instead of plt.figure(100); plt.clf()) is slower.

@eendebakpt eendebakpt mentioned this issue Jul 3, 2023
Closed
1 task
@eendebakpt
Copy link
Contributor Author

The method reset_ticks seems to be called a lot, also on ticks (xticks, yticks) that have already been reset. Adding this snippet

    import os
    from inspect import stack
    
    def fmt(f):
        bname = os.path.basename(f.filename)
        s=f'{bname}: line {f.lineno}: function {f.function}'
        s+=' '*max(0, 40-len(s))
        return f'{s} <-- context {f.code_context}'
    print("reset_ticks:")
    s=stack()
    for ii,f in enumerate(s[0: min(15, len(s))]):
              print(f'  {ii}.{fmt(f)}')

to reset_ticks in matplotlib/axis.py gives some information on where the reset_ticks is triggered.

Example output for script

plt.figure(100); 
print('# gca')
ax=plt.gca()
print('# clf')
plt.clf()

is

reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2.axis.py: line 699: function __init__     <-- context ['        self.clear()\n']
  3.deprecation.py: line 454: function wrapper <-- context ['        return func(*args, **kwargs)\n']
  4.axis.py: line 2220: function __init__    <-- context ['        super().__init__(*args, **kwargs)\n']
  5._base.py: line 777: function _init_axis  <-- context ['        self.xaxis = maxis.XAxis(self)\n']
  6._base.py: line 645: function __init__    <-- context ['        self._init_axis()\n']
  7._subplots.py: line 34: function __init__ <-- context ['        self._axes_class.__init__(self, fig, [0, 0, 1, 1], **kwargs)\n']
  8.figure.py: line 745: function add_subplot <-- context ['            ax = subplot_class_factory(projection_class)(self, *args, **pkw)\n']
  9.figure.py: line 1616: function gca       <-- context ['        return ax if ax is not None else self.add_subplot()\n']
  10.pyplot.py: line 2237: function gca       <-- context ['    return gcf().gca()\n']
  11.mpl_test.py: line 47: function <module>  <-- context ['ax=plt.gca()\n']
reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2.spines.py: line 217: function register_axis <-- context ['            self.axis.clear()\n']
  3._base.py: line 778: function _init_axis  <-- context ['        self.spines.bottom.register_axis(self.xaxis)\n']
  4._base.py: line 645: function __init__    <-- context ['        self._init_axis()\n']
  5._subplots.py: line 34: function __init__ <-- context ['        self._axes_class.__init__(self, fig, [0, 0, 1, 1], **kwargs)\n']
  6.figure.py: line 745: function add_subplot <-- context ['            ax = subplot_class_factory(projection_class)(self, *args, **pkw)\n']
  7.figure.py: line 1616: function gca       <-- context ['        return ax if ax is not None else self.add_subplot()\n']
  8.pyplot.py: line 2237: function gca       <-- context ['    return gcf().gca()\n']
  9.mpl_test.py: line 47: function <module>  <-- context ['ax=plt.gca()\n']
reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2.spines.py: line 217: function register_axis <-- context ['            self.axis.clear()\n']
  3._base.py: line 779: function _init_axis  <-- context ['        self.spines.top.register_axis(self.xaxis)\n']
  4._base.py: line 645: function __init__    <-- context ['        self._init_axis()\n']
  5._subplots.py: line 34: function __init__ <-- context ['        self._axes_class.__init__(self, fig, [0, 0, 1, 1], **kwargs)\n']
  6.figure.py: line 745: function add_subplot <-- context ['            ax = subplot_class_factory(projection_class)(self, *args, **pkw)\n']
  7.figure.py: line 1616: function gca       <-- context ['        return ax if ax is not None else self.add_subplot()\n']
  8.pyplot.py: line 2237: function gca       <-- context ['    return gcf().gca()\n']
  9.mpl_test.py: line 47: function <module>  <-- context ['ax=plt.gca()\n']
reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2.axis.py: line 699: function __init__     <-- context ['        self.clear()\n']
  3.deprecation.py: line 454: function wrapper <-- context ['        return func(*args, **kwargs)\n']
  4.axis.py: line 2471: function __init__    <-- context ['        super().__init__(*args, **kwargs)\n']
  5._base.py: line 780: function _init_axis  <-- context ['        self.yaxis = maxis.YAxis(self)\n']
  6._base.py: line 645: function __init__    <-- context ['        self._init_axis()\n']
  7._subplots.py: line 34: function __init__ <-- context ['        self._axes_class.__init__(self, fig, [0, 0, 1, 1], **kwargs)\n']
  8.figure.py: line 745: function add_subplot <-- context ['            ax = subplot_class_factory(projection_class)(self, *args, **pkw)\n']
  9.figure.py: line 1616: function gca       <-- context ['        return ax if ax is not None else self.add_subplot()\n']
  10.pyplot.py: line 2237: function gca       <-- context ['    return gcf().gca()\n']
  11.mpl_test.py: line 47: function <module>  <-- context ['ax=plt.gca()\n']
reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2.spines.py: line 217: function register_axis <-- context ['            self.axis.clear()\n']
  3._base.py: line 781: function _init_axis  <-- context ['        self.spines.left.register_axis(self.yaxis)\n']
  4._base.py: line 645: function __init__    <-- context ['        self._init_axis()\n']
  5._subplots.py: line 34: function __init__ <-- context ['        self._axes_class.__init__(self, fig, [0, 0, 1, 1], **kwargs)\n']
  6.figure.py: line 745: function add_subplot <-- context ['            ax = subplot_class_factory(projection_class)(self, *args, **pkw)\n']
  7.figure.py: line 1616: function gca       <-- context ['        return ax if ax is not None else self.add_subplot()\n']
  8.pyplot.py: line 2237: function gca       <-- context ['    return gcf().gca()\n']
  9.mpl_test.py: line 47: function <module>  <-- context ['ax=plt.gca()\n']
reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2.spines.py: line 217: function register_axis <-- context ['            self.axis.clear()\n']
  3._base.py: line 782: function _init_axis  <-- context ['        self.spines.right.register_axis(self.yaxis)\n']
  4._base.py: line 645: function __init__    <-- context ['        self._init_axis()\n']
  5._subplots.py: line 34: function __init__ <-- context ['        self._axes_class.__init__(self, fig, [0, 0, 1, 1], **kwargs)\n']
  6.figure.py: line 745: function add_subplot <-- context ['            ax = subplot_class_factory(projection_class)(self, *args, **pkw)\n']
  7.figure.py: line 1616: function gca       <-- context ['        return ax if ax is not None else self.add_subplot()\n']
  8.pyplot.py: line 2237: function gca       <-- context ['    return gcf().gca()\n']
  9.mpl_test.py: line 47: function <module>  <-- context ['ax=plt.gca()\n']
reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2._base.py: line 1236: function __clear    <-- context ['            axis.clear()  # Also resets the scale to linear.\n']
  3._base.py: line 1350: function clear      <-- context ['            self.__clear()\n']
  4._base.py: line 653: function __init__    <-- context ['        self.clear()\n']
  5._subplots.py: line 34: function __init__ <-- context ['        self._axes_class.__init__(self, fig, [0, 0, 1, 1], **kwargs)\n']
  6.figure.py: line 745: function add_subplot <-- context ['            ax = subplot_class_factory(projection_class)(self, *args, **pkw)\n']
  7.figure.py: line 1616: function gca       <-- context ['        return ax if ax is not None else self.add_subplot()\n']
  8.pyplot.py: line 2237: function gca       <-- context ['    return gcf().gca()\n']
  9.mpl_test.py: line 47: function <module>  <-- context ['ax=plt.gca()\n']
reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2._base.py: line 1236: function __clear    <-- context ['            axis.clear()  # Also resets the scale to linear.\n']
  3._base.py: line 1350: function clear      <-- context ['            self.__clear()\n']
  4._base.py: line 653: function __init__    <-- context ['        self.clear()\n']
  5._subplots.py: line 34: function __init__ <-- context ['        self._axes_class.__init__(self, fig, [0, 0, 1, 1], **kwargs)\n']
  6.figure.py: line 745: function add_subplot <-- context ['            ax = subplot_class_factory(projection_class)(self, *args, **pkw)\n']
  7.figure.py: line 1616: function gca       <-- context ['        return ax if ax is not None else self.add_subplot()\n']
  8.pyplot.py: line 2237: function gca       <-- context ['    return gcf().gca()\n']
  9.mpl_test.py: line 47: function <module>  <-- context ['ax=plt.gca()\n']
# clf
reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2._base.py: line 1236: function __clear    <-- context ['            axis.clear()  # Also resets the scale to linear.\n']
  3._base.py: line 1350: function clear      <-- context ['            self.__clear()\n']
  4.figure.py: line 953: function clear      <-- context ['            ax.clear()\n']
  5.figure.py: line 3044: function clear     <-- context ['        super().clear(keep_observers=keep_observers)\n']
  6.pyplot.py: line 933: function clf        <-- context ['    gcf().clear()\n']
  7.mpl_test.py: line 65: function <module>  <-- context ['plt.clf()\n']
reset_ticks:
  0.axis.py: line 929: function reset_ticks  <-- context ['                s=stack()\n']
  1.axis.py: line 897: function clear        <-- context ['        self.reset_ticks()\n']
  2._base.py: line 1236: function __clear    <-- context ['            axis.clear()  # Also resets the scale to linear.\n']
  3._base.py: line 1350: function clear      <-- context ['            self.__clear()\n']
  4.figure.py: line 953: function clear      <-- context ['            ax.clear()\n']
  5.figure.py: line 3044: function clear     <-- context ['        super().clear(keep_observers=keep_observers)\n']
  6.pyplot.py: line 933: function clf        <-- context ['    gcf().clear()\n']
  7.mpl_test.py: line 65: function <module>  <-- context ['plt.clf()\n']

@timhoffm
Copy link
Member

timhoffm commented Sep 9, 2024 •
edited
Loading

To be checked: Why do we clear the Axes before deleting them?

matplotlib/lib/matplotlib/figure.py

Line 987 in 5461ae2

ax.clear()

Axes.clear() is expensive because of the tick logic. But resetting ticks is not necessary when we remove the Axes anyway. We should try and remove that ax.clear() call. It might be that this call was introduces to cleans up some additional stuff, but OTOH if that's the case it should be handled in delaxes() (delaxes is a public API and should work with any Axes, not just cleared ones).

@tacaswell
Copy link
Member

My off-the-cuff guess is that we are doing that to help the garbage collector out and break some of the reference cycles.

@timhoffm
Copy link
Member

timhoffm commented Sep 9, 2024

Ok, then we could possibly write some more lightweight prepare_del() (better name t.b.d.) function that untangles an Axes from all relations without resetting internal state unnecessarily.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Performance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tacaswell @eendebakpt @timhoffm @oscargus