Description
The current implementation of shrinkage 'auto' is to calculate ledoit-wolf shrinkage factor separately for each matrix.
The within-class covariance matrices are calculated with different shrinkage factors, then summed together to be S_w. Because shrinkage factor depends largely on sample size, the shrinkage factor of every within-class covariance matrix will be much larger than that of the total covariance matrix, meaning diagonal entries are weighted more in S_w than in S_t. This will result in negative diagonal entries for the between-class covariance matrix $S_b=S_t-S_w$
negative variance then leads to weird behavior of eigen-value decomposition, including 1) wrong/meaningless explained_variance 2) questionable eigenvectors
this problem can be resolved by calculating the ledoit-wolf shrinkage factor first and pass it as an argument to the LDA decoder.
I'm not sure whether this is an intended feature or a bug.
If it's the former case, how should we interpret the eigenvectors?
Description
The current implementation of shrinkage 'auto' is to calculate ledoit-wolf shrinkage factor separately for each matrix.
The within-class covariance matrices are calculated with different shrinkage factors, then summed together to be S_w. Because shrinkage factor depends largely on sample size, the shrinkage factor of every within-class covariance matrix will be much larger than that of the total covariance matrix, meaning diagonal entries are weighted more in S_w than in S_t. This will result in negative diagonal entries for the between-class covariance matrix$S_b=S_t-S_w$
negative variance then leads to weird behavior of eigen-value decomposition, including 1) wrong/meaningless explained_variance 2) questionable eigenvectors
this problem can be resolved by calculating the ledoit-wolf shrinkage factor first and pass it as an argument to the LDA decoder.
I'm not sure whether this is an intended feature or a bug.
If it's the former case, how should we interpret the eigenvectors?