it would probably only be possible to compute the robustly estimated variance and mean up to some precision epsilon

Nitpicking and to avoid any confusion: RobustScaler does not estimate the mean and variance but rather the median and IQR. Those can be very different values depending on the distribution.

Personally, I find the reservoir sampling approach interesting. Not that a subsample param similar to what we do in KBinsDiscretizer would make sense, irrespective of whether or not we implement partial_fit. It could later be reused to control the size of the reservoir sampler used for partial_fit.

In addition to the methods referenced in the paper linked above, there is also the t-digest out-of-core / online estimator: https://github.com/tdunning/t-digest (but I have no idea about what are the pros and cons).

Not sure whether this should be considered a new algorithm or not, let me know what you think.

We could introduce an extra algorithm or quantile_method constructor parameter for the RobustScaler class to select the strategy.

Note that in #29907, we are introducing the quantile_method="averaged_inverted_cdf" parameter as the future default method to be able to implement the expected weight/repetition semantics when fitting KBinsDiscretizer with strategy="quantile" and sample_weight.

We can problem reuse that parameter in RobustScaler (and also add support for sample_weight). This parameter could then be expanded to implement quantile_method="t-digest" or similar later, and some of those can be made to support partial_fit (with or without subsample).

Thinking about it, the reservoir sampling approach might not be robust if the data is not i.i.d. or exchangeable, and might as a result lead to very different values that depend on the ordering of the data passed to successive partial_fit calls. Other methods are probably more robust to this problem.