I get

RidgeCV(eigen)
alpha: 200.0
cv: [ 78.16301385  78.56061227  78.95707779  79.35240503  79.7465887 ]
coef: [-0.90268438  1.03901964  0.0881258   0.68583855 -2.00008704]
RidgeCV(svd)
alpha: 200.0
cv: [ 78.16301385  78.56061227  78.95707779  79.35240503  79.7465887 ]
coef: [-0.90268438  1.03901964  0.0881258   0.68583855 -2.00008704]
Ridge
coef: [-0.10848951  0.36202508 -0.34585975  0.23836357 -0.28916756]

on master.
Not sure why it is different. But also not good.

First thought it was related to #4781 but maybe it isn't

ping @eickenberg?

FYI I am using 0.16.1. Could be dup of #4490 ?

I will take a look at this during this afternoon

Looking into this. If you pass in cv=N (so that it does standard cross validation, it works). The issue is down in the _RidgeGCV class, and only shows up with weights.

Simplified code to show issue:

import numpy as np
from sklearn.linear_model import RidgeCV, Ridge
from sklearn.datasets import load_boston
from sklearn.preprocessing import scale

boston = scale(load_boston().data)
target = load_boston().target
alphas = [.1]

for desc, weights in (('UNWEIGHTED', np.linspace(1, 1, len(target))),
                      ('WEIGHTED', np.linspace(1, 2, len(target)))):
    print('\n\n==== {} ====='.format(desc))
    print("---With CV---")
    ridge_cv = (RidgeCV(alphas=alphas, gcv_mode='eigen')
                .fit(boston, target, sample_weight=weights))
    print("alpha:", ridge_cv.alpha_)
    print("coef:", ridge_cv.coef_[:5])

    print("---Without CV---")
    ridge = (Ridge(ridge_cv.alpha_)
             .fit(boston, target, sample_weight=weights))
    print("alpha:", ridge.alpha)
    print("coef:", ridge.coef_[:5])

    diff = np.mean(np.abs(ridge_cv.predict(boston) - ridge.predict(boston)))
    print("\nMEAN DIFF: ", diff)

Output

==== UNWEIGHTED =====
---With CV---
nonezo
alpha: 0.1
coef: [-0.91956882  1.07943945  0.14054342  0.6825521  -2.05727602]
---Without CV---
alpha: 0.1
coef: [-0.91956882  1.07943945  0.14054342  0.6825521  -2.05727602]

MEAN DIFF:  6.41791407019e-12


==== WEIGHTED =====
---With CV---
nonezo
alpha: 0.1
coef: [-0.91863371  1.07728128  0.13826842  0.68280656 -2.05499068]
---Without CV---
alpha: 0.1
coef: [-0.91956882  1.07943945  0.14054342  0.6825521  -2.05727602]

sample_weights for RidgeGCV has been broken since the beginning. They
weight the eigenspaces of the gram matrix. I made a PR to fix it, but dont
have access to github right now (phone) to find the number

On Monday, November 16, 2015, Justin Vincent [email protected]
wrote:

Looking into this. If you pass in cv=N (so that it does standard cross
validation, it works). The issue is down in the _RidgeGCV class, and only
shows up with weights.

Simplified code to show issue:

import numpy as npfrom sklearn.linear_model import RidgeCV, Ridgefrom sklearn.datasets import load_bostonfrom sklearn.preprocessing import scale

boston = scale(load_boston().data)
target = load_boston().target
alphas = [.1]
for desc, weights in (('UNWEIGHTED', np.linspace(1, 1, len(target))),
('WEIGHTED', np.linspace(1, 2, len(target)))):
print('\n\n==== {} ====='.format(desc))
print("---With CV---")
ridge_cv = (RidgeCV(alphas=alphas, gcv_mode='eigen')
.fit(boston, target, sample_weight=weights))
print("alpha:", ridge_cv.alpha_)
print("coef:", ridge_cv.coef_[:5])

print("---Without CV---")
ridge = (Ridge(ridge_cv.alpha_)
         .fit(boston, target, sample_weight=weights))
print("alpha:", ridge.alpha)
print("coef:", ridge.coef_[:5])

diff = np.mean(np.abs(ridge_cv.predict(boston) - ridge.predict(boston)))
print("\nMEAN DIFF: ", diff)

Output

==== UNWEIGHTED =====
---With CV---
nonezo
alpha: 0.1
coef: [-0.91956882 1.07943945 0.14054342 0.6825521 -2.05727602]
---Without CV---
alpha: 0.1
coef: [-0.91956882 1.07943945 0.14054342 0.6825521 -2.05727602]

MEAN DIFF: 6.41791407019e-12

==== WEIGHTED =====
---With CV---
nonezo
alpha: 0.1
coef: [-0.91863371 1.07728128 0.13826842 0.68280656 -2.05499068]
---Without CV---
alpha: 0.1
coef: [-0.91956882 1.07943945 0.14054342 0.6825521 -2.05727602]

—
Reply to this email directly or view it on GitHub
#5364 (comment)
.

#4490 fixes this:

==== UNWEIGHTED =====
---With CV---
alpha: 0.1
coef: [-0.91956882  1.07943945  0.14054342  0.6825521  -2.05727602]
---Without CV---
alpha: 0.1
coef: [-0.91956882  1.07943945  0.14054342  0.6825521  -2.05727602]

MEAN DIFF:  6.41791407019e-12


==== WEIGHTED =====
---With CV---
alpha: 0.1
coef: [-0.9281295   1.17051021  0.08689879  0.77151052 -2.31879632]
---Without CV---
alpha: 0.1
coef: [-0.9281295   1.17051021  0.08689879  0.77151052 -2.31879632]

MEAN DIFF:  1.21976561482e-11

There is a small difference in between Ridge and RidgeCV which is cross-validation. Normal Ridge doesn't perform cross validation but whereas the RidgeCV will perform Leave-One-Out cross-validation even if you give cv = None(Node is taken by default). Maybe this is why they produce a different set of results.