Indeed we relay on matplotlib to recognize the plotted figures and save them. We have an additional support for mayavi plots too.
Which software do you use? I would like to see your minimal example.
And then we'll have to think about this. And how to support other plotting utilities. Like we do with mayavi including it in the main flow of the script or try to make implement the capture of images as loadable plugins.

Here's a simple yt script that demonstrates the issue:

import yt
from yt.testing import fake_random_ds

ds = fake_random_ds()

plot = yt.SlicePlot(ds, 'z', 'density')

plot.save()

And here's another script that uses matplotlib (this is more or less what yt is doing internally)

from matplotlib import Figure
from matplotlib.figure import Figure
from matplotlib.backends.backend_agg import FigureCanvasAgg

fig = Figure()
ax = fig.add_subplot(111)
ax.plot([1,2,3])
canvas = FigureCanvasAgg(fig)
canvas.print_png('image.png')

This doesn't use the pyplot state machine at all, so the machinery in sphinx-gallery that detects figures produced by example scripts isn't aware of the figure I've saved to disk.

The approach I was planning to implement was to add a new configuration option to look for new png images produced by a script. If this configuration option is turned on, sphinx-gallery will search for images produced by a script and include them in the gallery in a manner analogous to the current handling of pyplot-generated figures. This would probably be implemented as an additional if clause below the one that handles mayavi figures.

In addition, what if someone wanted to make a sphinx-gallery for a library like Pillow that generates png images completely independently of matplotlib? They'd need something like this to be able to include their images in a gallery.

In my mind the assumption of sphinx-gallery is that you run your example and you see figures pop up. The images in the rendered HTML should be the figures that pop up.

yt purposely doesn't support interactive plotting (since it is commonly used in headless sessions). Rather than a plot popping up, it is saved to disk. From my perspective there is very little difference, especially when a script pretty clearly calls a function named "save".

Out of interest, do you have a link to your project gallery, mostly to have an idea how it looks and the kind of examples you have?

Not yet no, I'm still working on converting it. Our existing cookbook is here:

http://yt-project.org/docs/dev/cookbook/index.html

Here's a gif showing what I have so far. This was generated using a version of sphinx-gallery with PR #208 applied: http://imgur.com/a/evidx

This is related to a conversation I had with some of the pycortex people, who were in need of functionality to grab rst image directives in lieu of any interactively plotted viz. E.g., if you wanted to demo something that was inherently interactive but included a sample screenshot in the demo by linking to a PNG file, that image would be shown in the gallery if no other pyplot calls were made.

Just a note that #208 was closed because we didn't reach a consensus on whether this should be supported...are just deciding not to support this? Or is it worth somebody trying to finish up that PR?

It would be really nice to support this. Not all graphics produced via python happen via pyplot. Encoding that assumption into sphinx-gallery means many libraries won't be able to use this project at all for their documentation.

The approach I was planning to implement was to add a new configuration option to look for new png images produced by a script.

I think the most general approach would be to allow custom post-block image scrapers that take in which code block was run (maybe the leading comments, and the code?) and returns a list of image arrays (which SG will then save to the correct filenames).

The simplest implementation of this could be a new class LocalImageScraper that e.g. searches the local directory for new images, and adds those to the list.

Another implementation could e.g. look for a VisPy canvas and take a screenshot of that (vispy/vispy#557).

So I think it's pretty general, and allows people to extend SG beyond matplotlib and mayavi. These could also even be turned into such plugins. Maybe a config var like:

    plot_scrapers=['matplotlib', 'mayavi', LocalImageScraper, VisPyScraper]

where the str automatically trap to the classes that SG already implements (well, after they are refactored to be classes rather than being embedded directly in the code), and LocalImageScraper and VisPyScraper are a custom classes implemented by users.

(I'm volunteering to do this refactoring / new config var if people agree it makes sense)

Just curious, do you have a use case for this that is more precise that "people will want to do that at one point"?

The thing I am a bit worried a bit is that the test coverage of sphinx-gallery is not that great so it is easy to break small things without realising. See #301 for example where we got bitten by this recently.

About capturing saved files, this is what I had to say at the time:
#208 (comment) and probably #208 (comment). Mostly they should not have the same look as matplotlib figures to reduce confusion.

A related question: people increasingly ask me if sphinx-gallery will be able to handle HTML output / javascript viz libraries. There are more and more of these popping up (e.g. Altair, Bokeh, ipyvolume, ipywidgets, etc). Riffing off of @larsoner 's thoughts maybe it could be an HTMLImageScraper or something?

How technically hard is it?

That is the big question :-) I am not sure off the top of my head but happy to brainstorm about it. Maybe that'd be worth its own separate issue though as it's a bit tangential to scraping files saved to disk?

I was chatting w/ @jakevdp about this and he was interested in whether sphinx-gallery could be used w/ Altair. Maybe he has thoughts about how the outputs of these kinds of packages could be collected etc.

Just curious, do you have a use case for this that is more precise that "people will want to do that at one point"?

Yes, in addition to the image-saving that this issue talks about, I'd like to use SG for VisPy as linked above: vispy/vispy#557

That is the big question :-) I am not sure off the top of my head but happy to brainstorm about it. Maybe that'd be worth its own separate issue though as it's a bit tangential to scraping files saved to disk?

It doesn't sound too terrible (look for <img tags and parse, hopefully), but in any case it fits in with the framework I proposed just fine

Building a list of capturers, and at the same time we must have a plugin list for their representation, not everything will be an image.

This is a much more extensive proposal than what I have in mind. You're basically talking about changing SG from a run/extract-image/make-pretty library to run/extract-anything/make-pretty library. This clearly has more potential flexibility, but I suspect will be a lot more difficult to do and get right. Take a look at the (relative) simplicity of #313, which just does image_scrapers. Hopefully that's good enough as incremental progress...

Closing this as the use case seems to be addressed by our new scraping API, feel free to reopen if I'm mistaken @ngoldbaum