FigureCanvasTkAgg memory leak #24820

cenfra-git · 2022-12-27T00:17:56Z

Bug summary

I'm trying to update a plot in tkinter by destroying the frame where the FigureCanvasTkAgg tk widget is packed and creating another instance of FigureCanvasTkAgg, but the memory of the script only goes up after repeating the process several times.

Code for reproduction

import tkinter as tk
from tkinter import ttk
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
from matplotlib.figure import Figure
import numpy as np

class Main_Program(tk.Tk):
    def __init__(self, *args, **kwargs):
        tk.Tk.__init__(self, *args, **kwargs)
        
        self.geometry("500x500")
        
        # ---------------------------------------- TOP FRAME (Button)
        
        self.top_frame = tk.Frame(self)  
        ttk.Button(self.top_frame,text="Update graph",command=self.update_graph).pack()
        self.top_frame.pack()
        
        # ---------------------------------------- BOTTOM FRAME (Graph)
        
        self.bottom_frame = tk.Frame(self) 
         
        DisplayGraph(self.bottom_frame,
        x_limit= (-10, 10) ,
        delta_x= 0.1 ,
        y_input= "2*x+2" )
        
        self.bottom_frame.pack() 
        
    
    def update_graph(self):
        
        # ----------------------------- CLEAR
        
        for widget in self.bottom_frame.winfo_children():
            widget.destroy()
        
        #  ----------------------------- DISPLAY NEW GRAPH
        
        DisplayGraph(self.bottom_frame,
        x_limit= (-10, 10) ,
        delta_x= 0.1 ,
        y_input= "2*x+2" )
        
class DisplayGraph:
    def __init__(self,passed_frame,
                x_limit=(), # TUPLE (-10,10)
                delta_x=0.0, # FLOAT 0.1
                y_input='', # STRING 2*x+2
                figsize_x=6,
                figsize_y=5):

        self.x_limit = x_limit
        self.delta_x = delta_x
        self.y_input = y_input
        
        # ---------------------------------------------------------- CREATE FIGURE AND SUBPLOT

        self.fig = Figure(figsize=(figsize_x,figsize_y),dpi=100)
        self.ax = self.fig.add_subplot(111)
        self.ax.grid()
        
        # ---------------------------------------------------------- PLOT
        
        self.plot_input()

        # ---------------------------------------------------------- PACK AND DRAW

        self.canvas_frame = tk.Frame(passed_frame,bg="black")
        self.canvas_mat = FigureCanvasTkAgg(self.fig, master=self.canvas_frame)
        
        self.canvas_mat.get_tk_widget().pack(anchor="center", expand=True,padx=2,pady=2)
        self.canvas_frame.pack(side=tk.LEFT,anchor="center", expand=True)
        
        self.canvas_mat.draw()
        
        # ----------------------------------------------------------
        
    def plot_input(self):

        x = np.arange(self.x_limit[0],self.x_limit[1],self.delta_x)
        y = eval(self.y_input)

        self.ax.plot(x,y)

App = Main_Program()
App.mainloop()

Actual outcome

Memory after running the script for the first time

Memory after clicking the button 20 times:

I've seen that some of the memory is freed after a while but not immediately after updating the frame.

Expected outcome

I would expect that destroying the frame where the FigureCanvasTkAgg widget is located would allow me to create another instance of FigureCanvasTkAgg without increasing the memory (or at least too much).

Additional information

I've seen posts as far back as 13 years ago reporting the issue in different places, but I have not been able to find a way to solve it. I know that you can create only one instance of the FigureCanvasTkAgg object and clear/plot each time, but I'm working on a project that involves several figures at the same time and closing the tkinter toplevel window does not seem to clear the memory.

Operating system

Windows

Matplotlib Version

3.6.2

Matplotlib Backend

TkAgg

Python version

3.11.0

Jupyter version

No response

Installation

pip

richardsheridan · 2023-01-27T18:54:31Z

This version of the tk memory leak is even more fundamental than some of the other open issues, because it doesn't involve FigureManagerTk at all.

If we write the example in a more conventional tkinter style:

Longish code

import itertools
import tkinter as tk
from tkinter import ttk

from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
from matplotlib.figure import Figure
import numpy as np

counter = itertools.count()


def DisplayGraph(
    passed_frame, x_limit=(), delta_x=0.0, y_input="", figsize_x=6, figsize_y=5
):
    fig = Figure(figsize=(figsize_x, figsize_y), dpi=100)
    ax = fig.add_subplot(111)
    ax.grid()
    x = np.arange(x_limit[0], x_limit[1], delta_x)
    y = eval(y_input)
    ax.plot(x, y)
    canvas_mat = FigureCanvasTkAgg(fig, master=passed_frame)
    canvas_mat.get_tk_widget().pack(anchor="center", expand=True, padx=2, pady=2)
    canvas_mat.count = counter.__next__()
    # canvas_mat.draw()  #  redundant!


def update_graph():
    for widget in bottom_frame.winfo_children():
        widget.destroy()
    DisplayGraph(bottom_frame, x_limit=(-10, 10), delta_x=0.1, y_input="2*x+2")
    root.after_idle(report)


def report():
    import gc, psutil, objgraph

    print(gc.collect())

    def filt(o):
        return isinstance(o, FigureCanvasTkAgg)

    objgraph.show_growth(filter=filt)
    objgraph.show_backrefs(
        [x for x in gc.get_objects() if filt(x) and not x.count],
        max_depth=10,
        filename="stuff.png",
    )
    print(psutil.Process().memory_full_info().uss)


root = tk.Tk()
root.geometry("500x500")
top_frame = tk.Frame(root)
ttk.Button(top_frame, text="Update graph", command=update_graph).pack()
top_frame.pack()
bottom_frame = tk.Frame(root)
DisplayGraph(bottom_frame, x_limit=(-10, 10), delta_x=0.1, y_input="2*x+2")
bottom_frame.pack()

root.mainloop()

We can clearly see that there are no references to the figure or canvas left after leaving the `DisplayGraph` function, so you'd think all matplotlib components would be gone before tkinter has a chance to render them. Yet, the interactive updates of the resize callback are still working, so they have not been garbage collected. We can see that instances of `FigureCanvasTkAgg` are surviving the garbage collection and growing by one instance each cycle, which just about accounts for the size of the memory leak.

It's no secret that there is a reference cycle here, namely fig.canvas <-> canvas.figure, but the confusing thing is how they are surviving gc.collect. Objgraph (isolating the first-ever FigureCanvasTkAgg) points to two culprits. First, the canvas is referenced in the MouseEvent sitting in LocationEvent.lastevent, which is a long-lived, class-attribute reference in matplotlib.backend_bases:

Big object graph image

(likely better viewed in its own window. Look for the red arrow!)

It would be better if this reference didn't live so long, but we can overwrite it by clicking somewhere in the plot before hitting the update button again. This cleans up the graph and reveals the only other things holding FigureCanvasTkAgg alive:

Less big object graph image

From the perspective of the python object graph, these are "leaks" since they have no referents. but what it actually means in this specific case is that the references are "owned" by the tkinter c code.

Fortunately, we can see which methods are referenced by the leaks specifically, directing us to the culprit binds:

matplotlib/lib/matplotlib/backends/_backend_tk.py

Lines 201 to 208 in 84cc898

    
           root.bind("<MouseWheel>", self.scroll_event_windows, "+") 
        
           # Can't get destroy events by binding to _tkcanvas. Therefore, bind 
        
           # to the window and filter. 
        
           def filter_destroy(event): 
        
               if event.widget is self._tkcanvas: 
        
                   CloseEvent("close_event", self)._process() 
        
           root.bind("<Destroy>", filter_destroy, "+")

These are not cleaned up by the widget.destroy() logic because they are bound to the toplevel window rather than the canvas. If you comment them out, the leaks shrink by 99%.

But at this point, I'm stuck, because we sorta need those?! and it's not obvious how we could sneak an unbind into the logic, at least not without having users make an extra method call they never had to make before.

CC @tacaswell @anntzer @oscargus

tacaswell · 2023-01-27T19:00:46Z

Can we use a weakref in the callbacks rather than self?

richardsheridan · 2023-01-27T19:17:43Z

This works but it's a bit ugly? weakref.proxy gave various errors. is there a better way?

        root = self._tkcanvas.winfo_toplevel()
        weakself = weakref.ref(self)
        def scroll_event_windows(event):
            self = weakself()
            if self is None:
                return
            return self.scroll_event_windows(event)
        root.bind("<MouseWheel>", scroll_event_windows, "+")

        # Can't get destroy events by binding to _tkcanvas. Therefore, bind
        # to the window and filter.
        def filter_destroy(event):
            self = weakself()
            if self is None:
                return
            if event.widget is self._tkcanvas:
                CloseEvent("close_event", self)._process()
        root.bind("<Destroy>", filter_destroy, "+")

Also it would be good to perform an unbind rather than just short-circuiting the existing bind function.

tacaswell · 2023-01-27T19:28:15Z

un-bind as part of the short-circuit so we will run those at most once?

dstansby added the Performance label Dec 27, 2022

richardsheridan mentioned this issue Jan 27, 2023

fix FigureCanvasTkAgg memory leak via weakrefs #25097

Merged

6 tasks

anntzer mentioned this issue Jan 27, 2023

Deprecate LocationEvent.lastevent. #25101

Merged

6 tasks

ksunden closed this as completed in #25097 Feb 7, 2023

QuLogic added this to the v3.7.0 milestone Feb 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FigureCanvasTkAgg memory leak #24820

FigureCanvasTkAgg memory leak #24820

cenfra-git commented Dec 27, 2022

richardsheridan commented Jan 27, 2023 •

edited

Loading

tacaswell commented Jan 27, 2023

richardsheridan commented Jan 27, 2023

tacaswell commented Jan 27, 2023

FigureCanvasTkAgg memory leak #24820

FigureCanvasTkAgg memory leak #24820

Comments

cenfra-git commented Dec 27, 2022

Bug summary

Code for reproduction

Actual outcome

Expected outcome

Additional information

Operating system

Matplotlib Version

Matplotlib Backend

Python version

Jupyter version

Installation

richardsheridan commented Jan 27, 2023 • edited Loading

tacaswell commented Jan 27, 2023

richardsheridan commented Jan 27, 2023

tacaswell commented Jan 27, 2023

richardsheridan commented Jan 27, 2023 •

edited

Loading