I am interested in plotting very long time series and allow interactive viewing of the series (i.e., a 2d line plot where the x-coordinates of the points is np.arange(N), and the y-coordinates is the data). By long, I mean up to N=10**9, and ideally I'd like to plot a few of these series on the same plot. The plot should be interactive with the possibility of zooming from viewing the whole dataset down to viewing only a few points.

Conceptually, everything I need (and more!) is provided by LineCollection, but this use-case is pushing the limits and encountering some limitations. I therefore started developing a library to work around these limitations (https://github.com/cassiersg/scaviz), which I discuss below. Any comment/input is welcome. I am also interested in contributing upstream if some features are of interest to fastplotlib.

Performance

The first is performance: on my quick experiments with my iGPU, I see the frame-rate dropping (and overall high system resource usage) when going above ~10**6 points in total. This is expected since calling the vertex shader for that many points will be costly and won't scale well to larger datasets. I combined two solutions to solve this issue:

1. Do not render points that are not visible (that's easy to know, we simply select a range of data according to the part of the x axis that is visible). This tweak alone gives excellent performance when only relatively few points are visible, i.e., when zooming in a lot.
2. For zoomed-out views, there are multiple datapoints per pixel. It is inefficient to re-aggregate them when rendering each frame. Therefore, we can pre-compute down-sampled versions of the data and render that instead of the original data. In order to keep the plot visual appearance correct, when downsampling k points into one, we take the min and max of the k points, then color the whole (min, max) interval along the y axis for the corresponding x coordinate.

The current implementation does 1. using a custom shader which is very similar to pygfx's Line, but stores only the y coordinate buffer to save vRAM (z is a uniform and x is computed). For this to work, the vertex indices range that are rendered needs to change upon camera changes, which is currently done with a hack to pygfx (pygfx/pygfx#1078).

For 2., the pooling is done with numpy when creating the graphic and the pooled vectors are concatenated and transferred to the GPU. At every camera update, we select a "pooling scale" in the data, along with the rendered range and update uniform buffer with this information. The rest of the rendering is still to be done properly: currently I simply render a line that zig-zags between the min and max (this works reasonably well, but shows small artifacts).

Numerical issues

Given that 32-bit floating point numbers have 23 bits of mantissa, they have an effective precision of 24 bits: they can represent integers exactly up to 2**24-1, which is approx 16e6 and therefore lower than the maximum x coordinate I want to show. This limitation is not an issue in many use-cases since for most scientific data, a value of e.g. 10**9 or 10**9+50 doesn't make a difference, so the limited precision can be ignored.

However, for my usecase, when zooming in at the end of the sequence, it is actually important: when zooming on the end of the data that would mean that all points in a range [10**9, 10**9+50] get collapsed. In that case, the shader essentially computes x_orig = 10**9 (offset to start data rendering), delta = 20 (index of rendered point), then compute the world x coordinate x_orig+delta. Then, applying the camera transform essentially subtracts 10**9, giving the computation (10**9+20)-10**9, which will be incorrect.
I think that this can be fixed by setting x_orig=0 and passing to the shader a non-standard camera matrix whose position is the true camera position shifted by x_orig. The camera computations themselves are done in python and are lightweight so they can be done using 64-bit floats (maybe they already are), which solves the problem (with 52 mantissa, the data will not fit in any memory/storage before precision becomes an issue).

The same precision issues might appear for the axes/tick computations, and might need similar adaptation.

Other

• After adding a long sequence with add_graphic(..., center=True), zooming may end up in seeing only the background (no more axes of graphic visible), due to z=0 getting out of the camera's frustum (i.e., in front of the near plane or behind the far plane). This is a problem of numerical accuracy in the computation of the camera: while the x dimension is very large, the z range is small. Due to how show_object works when fov==0, we get the camera position very far from z=0 (approximately as far as the x dimension). The result of "large z distance" with "small z range" is that the z range gets shifted a bit. See Orthographic camera show_object distance. pygfx/pygfx#1081
• I made a simple sub-class of the PanZoomController to control separately the X and Y axes. I think that this is not strictly necessary since I could have one controller for each axis, however that makes the interface more complex: ideally, I'd like the API to be as nice to use as possible (same spirit as fastplotlib). I think that separating x and y zoom is a common need in scientific visualization, so maybe we can think about extending the upstream PanZoomController to support it (e.g., scrolling without modifier keys scales both axis, and modifier+scrolling for single axis zoom).
• The axis tick can be somewhat useless when zoomed in: e.g. when showing a range (10**5, 10**5+100), all the tick labels are 1e5, so we don't get any sense of the scale anymore. I didn't investigate this issue yet: it seems to me that the best solution is to dynamically adjust the precision of the ticks according to the displayed range, which actually looks like it is already done (I can see it going from 1 up to 4 significant digits when zooming, but then it stops: maybe there is a hard-coded threshold).