Add optional parameter to return raw data for diabetes dataset - #16605

The dataset description included here says that each column has been scaled and centred, which accounts for this difference. I agree that there would be benefit in allowing the user to retrieve the raw values. Apart from anything else, there is some data leakage in scaling across training and test data. But this should be done by providing a parameter to optionally return standardized values (default True to preserve current behaviour).

My bad, I completely missed that part from the description. In addition to the leakage issue, not all the columns (for ex gender) makes sense to be scaled and centered. I can augment this pr to return raw vs scaled / centered based on an optional parameter

Thanks @jnothman for feedback. I followed your suggestion and added an optional parameter standardized (defaults to True to preserve current behavior) which when set to False will load the raw data for the features.

Thanks! Could you please add a new test to check that the new option works as described?

For instance a new test that loads that when fetching the un-standardized data and standardizing our-selves we get a resulting data matrix that is close to the one we fetch when we call load_diabetes with standardized=True.

Thanks! Could you please add a new test to check that the new option works as described?

For instance a new test that loads that when fetching the un-standardized data and standardizing our-selves we get a resulting data matrix that is close to the one we fetch when we call load_diabetes with standardized=True.

I created a new test called load_diabetes_raw that checks for similar things as the other test for this dataset.

I would like to have a test that checks that the standardized values are what they are supposed to be but it does not seem to be the case. For instance the following test:

def test_load_diabetes_raw():
    diabetes_raw = load_diabetes(standardized=False)
    assert diabetes_raw.data.shape == (442, 10)
    assert diabetes_raw.target.size, 442
    assert len(diabetes_raw.feature_names) == 10
    assert diabetes_raw.DESCR

    diabetes_default = load_diabetes()

    np.testing.assert_allclose(
        scale(diabetes_raw.data),
        diabetes_default.data
    )

fails with:

    def test_load_diabetes_raw():
        diabetes_raw = load_diabetes(standardized=False)
        assert diabetes_raw.data.shape == (442, 10)
        assert diabetes_raw.target.size, 442
        assert len(diabetes_raw.feature_names) == 10
        assert diabetes_raw.DESCR
    
        diabetes_default = load_diabetes()
    
        n_samples = diabetes_raw.data.shape[0]
>       np.testing.assert_allclose(
            scale(diabetes_raw.data),
            diabetes_default.data
        )
E       AssertionError: 
E       Not equal to tolerance rtol=1e-07, atol=0
E       
E       Mismatched elements: 4420 / 4420 (100%)
E       Max absolute difference: 3.98049016
E       Max relative difference: 23.64621268
E        x: array([[ 0.8005  ,  1.065488,  1.297088, ..., -0.054499,  0.418531,
E               -0.370989],
E              [-0.039567, -0.938537, -1.08218 , ..., -0.830301, -1.436589,...
E        y: array([[ 0.038076,  0.05068 ,  0.061696, ..., -0.002592,  0.019908,
E               -0.017646],
E              [-0.001882, -0.044642, -0.051474, ..., -0.039493, -0.06833 ,...

I haven't had a deeper look but there is something wrong either in our data files or in the assumptions we make about them.

Ah, from the dataset description:

Each of these 10 feature variables have been mean centered and scaled by the standard deviation times `n_samples` (i.e. the sum of squares of each column totals 1)

So looks like we would need to divide the raw values by 442 ** 0.5 after scaling it.

Adjusting for precision loss, this test no longer fails:

    np.testing.assert_allclose(
        scale(diabetes_raw.data) / (442 ** 0.5),
        diabetes_default.data,
        atol=1e-04
    )

do we think it's necessary to distribute both diabetes_data_raw.csv.gz and diabetes_data.csv.gz, rather than just distributing one and scaling the data on the fly?

+1 for doing this.

I removed the (scaled) diabetes_data.csv.gz file and switched to relying on on the unscaled data diabetes_data_raw.csv.gzonly. This made a few docstests fail though, since the two versions of the data are only equal within a 1e-4 absolute tolerance. Fixed the corresponding doctests. In the end however, technically this would change slightly the exact values returned by load_diabetes .

I'm not sure whether we should go ahead (and consider this to be a fix) or not. In any case, I think shipping two versions of the data that's are not equal within float64 tolerance (which is much less that 1e-4) is not ideal.

If we only have 2 docstring failings, then I think that we can use the proposed option to rescale the dataset.
It seems better than using a CSV with truncated floating precision.

So all tests are passing now. I am merging. Thanks to all.