Somewhat curiously, with mplcairo.gtk (https://github.com/anntzer/mplcairo), this does not happen. (The initial draw is much slower though, and I think I know why; also the example; also this exposes a bug in mplcairo's scatter rendering that I need to look into. But I think the issue above is something about event loop integration so that doesn't matter.)

(EDIT: actually that's not a mplcairo bug but an inconsistency in matplotlib's own behavior, see #12021 -- mplcairo was drawing 4e6 points as would matplotlib's vector backends, whereas agg draws only 2e3.)

Looking at this more in depth, I think this is because the gtk3agg backend always rerenders the full figure in on_draw_event:

        if not len(self._bbox_queue):
            self._render_figure(w, h)

Compare with mplcairo.gtk which only does so if the last rendered buffer had been invalidated

        buf = self.get_renderer(_draw_if_new=True)._get_buffer()

(you need to look into the caching provided by _draw_if_new).

I think the builtin qt backends also avoid forcing a re-rendering in paintEvent through the draw_idle mechanism (which jumps through a lot of hoops), so it may be worth comparing it too.

This small modification seems does trick, but I not tested it in all use cases.

    def _render_figure(self, width, height):
        backend_agg.FigureCanvasAgg.draw(self)

    def queue_draw(self):    # ADDED
        allocation = self.get_allocation()
        w, h = allocation.width, allocation.height
        self._render_figure(w, h)
        super().queue_draw()

    def on_draw_event(self, widget, ctx):
        """ GtkDrawable draw event, like expose_event in GTK 2.X
        """
        allocation = self.get_allocation()
        w, h = allocation.width, allocation.height

        if not len(self._bbox_queue):
            # self._render_figure(w, h)  # REMOVED
            bbox_queue = [transforms.Bbox([[0, 0], [w, h]])]
        else:
            bbox_queue = self._bbox_queue

        if HAS_CAIRO_CFFI and not isinstance(ctx, cairo.Context):
            ctx = cairo.Context._from_pointer(
                cairo.ffi.cast('cairo_t **',
                               id(ctx) + object.__basicsize__)[0],
                incref=True)

Is this happening with the gtk3cairo backend?

@fariza Yes GTK3Cairo backend also redraws canvas same way. But same solution causes errors and I have no clue how to fix that errors.

Almost certainly the redraw can't be avoided with gtk3cairo, given that the whole idea of the backend is to draw the figure directly onto the cairo canvas that gtk's windowing system sets up for you, so there's no "previous" draw that can be kept around.