Coverage increased (+0.0%) when pulling db37aa6 on kpysniak:propag into 0dfa4d0 on scikit-learn:master.

Coverage increased (+0.0%) when pulling 027bf1f on kpysniak:propag into 0dfa4d0 on scikit-learn:master.

Alpha = 1 means hard clamping, right? Can you maybe add that to the docs? They are very rudimentary at the moment :-/

Do you think it makes sense to test for other values of alpha? I just looked around but it doesn't seem like the original publication had that parameter. Do you know which paper this is from?

I am not familiar with how clamping is suppose to work in LabelPropagation and co. Could you please improve the docstring for the parameters (in particular for alpha) to explain how they impact the model?

I think we need a reference. This PR basically inverts the meaning of alpha as far as I can tell. I have no way of knowing which one is the correct variant.

I think we need a reference. This PR basically inverts the meaning of alpha as far as I can tell. I have no way of knowing which one is the correct variant.

The narrative doc loosely defines the meaning of alpha and indeed it does not seem to be correct respected by the implementation. I agree that we need to compare our implementation with some official reference for the algorithm.

Maybe @clayw can give us insights into what alpha should mean and where it is defined in the literature.

It seems to be defined here:
http://citeseer.ist.psu.edu/viewdoc/download;jsessionid=FDD1A24102936B6FE8E329EB196E3BF4?doi=10.1.1.115.3219&rep=rep1&type=pdf
That is for label spreading, however, where alpha=1 doesn't seem to make sense.
Still looking for the parameter in label propagation.

I think LabelPropagation should not have a public alpha parameter, as it is not in the original publication and I can not find any other reference to it in this setting. So we should deprecate the public part and under the hood set it to zero. From the label spreading paper, it looks to me as if 0 (which is not allowed there) would be hard clamping.

It also looks to me like both algorithms can be implemented in closed form with single SVD (label spreading) or by solving a linear equation (label propagation), and there is no need to iterate.
Alpha in the label spreading paper is in ]0, 1[ and is the amount of information that flows. alpha=0 means don't spread any information, alpha=1 would be "replace all previous information" and both are not allowed. They use alpha=.99 in all experiments.

So todo:

• Fix the docs (that document alpha wrongly)
• Deprecate alpha in LabelPropagation (it should always be zero)
• implement the closed-form versions

Anyone interested or should I go for it?

Also, is there a reference for the word LabelSpreading? ^^

Thanks for running this investigation. Here are pictures from the Semi-Supervised Learning book that is referenced in our documentation. The algorithm is indeed iterative and it references the Zhou et al. paper.

Catching up here. That is indeed the original book referenced in the docs, and it looks like there's some solution planned out. Testing could be always be more thorough to catch these issues

Are there any action items that I could take here?

Huh, ok I only looked at the papers, not the book (where is it referenced?)
Is it this one?

Yoshua Bengio, Olivier Delalleau, Nicolas Le Roux. In Semi-Supervised Learning

I did not realize that was a reference to a book, I guess it is here:
http://www.acad.bg/ebook/ml/MITPress-%20SemiSupervised%20Learning.pdf

The papers have closed form algorithms.
I'll look at the book again.
@clayw the main action items would be to clarify the meaning of alpha (it is inconsistent between the docs and the code), clarify the meaning in LabelPropagation, and make it to be hard clamping by default.

Also, the citation should be

Bengio, Yoshua, Olivier Delalleau, and Nicolas Le Roux. "Label propagation and quadratic criterion." Semi-supervised learning 10 (2006).

Which would make it much much easier to find. I did not realize that was the reference.

@clayw the book also doesn't mention alpha for Label Propagation, right?

Alpha that's described in the literature for LabelPropagation in the book (only LabelSpreading), but I don't think it's such a crazy thing to support in this implementation. However, if there's no hard requirement to keep it around, then it can be deprecated.

Thanks for fixing the code to catch those cases and sprucing up the docs. Can you run the example at http://scikit-learn.org/stable/auto_examples/semi_supervised/plot_label_propagation_digits.html and verify that there aren't any performance issues with this change?

Hm, there is actually an alpha in Algorithm 11.2., is that what you were implementing?
Does that recover Algorithm 11.1 for a setting of alpha?

The docs say

The LabelPropagation algorithm performs hard clamping of input labels, which means \alpha=1. This clamping factor can be relaxed, to say \alpha=0.8, which means that we will always retain 80 percent of our original label distribution, but the algorithm gets to change it’s confidence of the distribution within 20 percent.

Can you point me to where this is explained in the paper?
link: http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_ssl.pdf

@amueller The three TODOs look like ideal tasks for me.

I'll open a new issue for the closed form version. We can further discuss there.
And I now start working on the two smaller TODOs.

Go for it :)
I think the most urgent part is making sure label propagation does hard clamping by default and that the documentation is consistent with the meaning of alpha.

Got it.

@kpysniak, do you plan to finish this PR?

@kpysniak, do you plan to finish this PR?

I think we can assume the answer is "no". Takers?

Can I take it up ? I am not an expert but will try my best. Thanks.

I have proposed a solution on #6727 months ago.

Hi, sorry for missing the link to the PR. Thanks for letting me know.

Great, @boechat107 sorry we missed it.

No worries, guys
;-)

Fixed in #9239