A simple workaround is to keep track of the sample mean itself, instead of the sample sum. Here a gist. However, there is caveat: the computation of the sample variance is less numerically accurate.

from sklearn.preprocessing.data import StandardScaler
import numpy as np

rng = np.random.RandomState(0)

def gen_1d_uniform_batch(min_, max_, n):
    return rng.uniform(min_, max_, size=(n, 1))

max_f = 1e15
min_f = max_f / 1e3
stream_dim = 10000
batch_dim = 1000
print("var divergence: batch vs online on %d repetitions" % stream_dim)

X = gen_1d_uniform_batch(min_=min_f, max_=min_f, n=batch_dim)

scaler = StandardScaler().fit(X)
iscaler = StandardScaler()

batch = gen_1d_uniform_batch(min_=min_f, max_=min_f, n=batch_dim)
for _ in range(stream_dim):
    iscaler = iscaler.partial_fit(batch)

print(scaler.var_)
[  0.]
print(iscaler.var_)
[  1.11203573e-05]

while on master the last print is 0.

Notice that while in the first code snippet I use large numbers to simulate the online stream, in the second case the lenght of the stream does not matter. It is the absolute value of the mean itself that is problematic. Indeed, if you run the second script with smaller max_f, everything is smooth. This is the offending step, which should trigger a potential catastrophic cancellation.

I think this means that we want this change in the code, but I would like to hear other opinions first. Maybe @jakevdp ?

I have no idea if other estimators with partial_fit may suffer the same issue. Maybe we could test for overflow for every partial_fit? #3907

We'd probably want to use something like Welford's algorithm to avoid overflow and precision loss in the case of partial_fit.

Started to take care of this in #11549

Is part of the issue that previously we were using Python ints and now we're using numpy ints?

This has been fixed already. The partial_fit documentation says it's been stabilized https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/preprocessing/_data.py#L879-L889

I can still reproduce using the OP's post.

@ogrisel recently was looking at a similar thing I think. Would you happen to know if this is solvable now?