I'm also wondering.

I played a bit with iris dataset

print(__doc__)

import matplotlib.pyplot as plt

from sklearn import datasets
from sklearn.decomposition import PCA
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis

iris = datasets.load_iris()

X = iris.data
X=X@np.diag([10,5,1,0.1])
#X= np.concatenate((X,X[:,0:2].mean(1,keepdims=True),np.zeros((150,2))),1)
y = iris.target
target_names = iris.target_names

ldal = LinearDiscriminantAnalysis(solver='lsqr', shrinkage=0.1)
ldal.fit(X,y)
print(ldal.score(X,y))
plt.imshow(ldal.covariance_)

lda = LinearDiscriminantAnalysis(solver='eigen', shrinkage=0.1)
lda.fit(X,y)
print(lda.score(X,y))
plt.figure()
plt.imshow(lda.covariance_)

By the implementation difference I mentioned above, as long as the covariance matrix is not identity matrix, there should be difference between these 2 solvers.

but it only gives different results in 2 scenarios:

1. variance for important features are very different and you use shrinkage
2. features are collinear and you use shrinkage

There seems to be something specific to shrinkage... I don't get it.

by changing self.coef_ from

self.coef_ = np.dot(self.means_, evecs).dot(evecs.T)

to

coef_=self.means_@np.linalg.inv(self.covariance_);

eigen solver reached the same performance as lsqr solver

can you propose a non-regression test?

don't we have tests to assess that results are the same across solvers? why is this not picked up? can you provide a non-regression test?

The reason the test passes is that it only checks that the predictions are the same and not that the posterior probabilities are equal. The test introduced in #11796 (test_lda_predict_proba) passes for svd and lsqr solvers, but fails for the eigen solver.

@lxx74n74n can you please confirm that the fix in #11796 solves this issue? Please feel free to jump in the discussion in that PR.