If I am not mistaken, I don't think that we have any example tackling the problem of recommendation. It would be nice to have a full example with a predictive model and the way to evaluate it?

take

@acse-srm3018 are you still working on this? If you have a draft, feel free to open a draft PR, mark it WIP, and ask for feedback if you need any help.

@adrinjalali Since nothing seems to happen, I would try it. I am quite new to scikit-learn though...could you tell me which file the new example should be in and what function(s) should be used in it?

You'd need to add a whole new example @sveneschlbeck , I wouldn't say it's the easiest issue to work on. You need to familiarize yourself with the structure of the examples under the examples/ folder, and add a new one in the appropriate place there.

@adrinjalali Thanks for the explanation. If I get you correctly, the example should be an example of recommendations in general or just the ranking metrices?

@adrinjalali Added a starter example (I myself used some time back) about a simple content-based movie rec engine using scikit-learn in the new dir examples/recommendation alongside a sample dataset.
The example also uses cosine similarity as a metric to rank the similarity of movies...

I don't think that we have any example tackling the problem of recommendation. It would be nice to have a full example with a predictive model and the way to evaluate it?

As far as I know ranking is not necessarily a primary evaluation metric in recommendation (see e.g. lightfm.evaluation): one cares more about how many relevant predictions are made in the first N, instead of whether it's first or third. Though I agree that it would still be be good to have a recommendation example, but maybe more for top_k_accuracy_score metric (which also doesn't have any examples apparently)?

For DCG and NDCG what comes to mind is more a search or directly a ranking problem. There are a few ranking problems on OpenML maybe we could pick one (or find some other open dataset and put it there)? Though of course we can also illustrate them on a recommendation example.

A side comment that https://www.openml.org/d/40916 looks interesting, but maybe too political. I do wonder how does the partial dependence plot of "Dystopia" wrt "Happines" looks like :)

@rth I agree...since having a Recommendation Engine example seemed to be in the interest of multiple people, I got to that first. The other points are also valid but (as you mentioned) not necessarily well-combineable with rec engines

Hi!

Happy to pick this up.

Just wanted to quickly confirm the direction before jumping in.

It seems like there are actually two separate things we have here, and maybe it’s worth splitting this into 2 different issues:

1. A recommendation example (something simple and educational, maybe video/movie based using cosine similarity) - this would flow more naturally into top_k_accuracy_score, which is usually what readers might have in mind and expect. Right now, k score doesn’t have any detailed example or walkthrough either, so we could build something that covers both.

2. A pure ranking example - where it makes more sense to use kendall or spearman for evaluation. These metrics are usually used in search, not so much in recommenders so a separate item might make sense.

My suggestion: I could start with the recommender + top_k_accuracy_score example first, since it would be what many users might expect when they come to this topic.

After that, I can follow up with a ranking example using kendall and spearman.

Let me know what direction makes more sense - happy to align on whatever would be most valuable for the readers!

@StefanieSenger

Let's tag @lorentzenchr here, so he can evaluate your approach.

@StefanieSenger in the meantime I will get started with laying down a rough structure for the example, dataset etc. (stuff that would remain slightly constant irrsepective of the approach we take for this one)

Hey, just getting this thread going again

@shivamchhuneja feel free to fire up a pull request with your ideas and we can take the idea from there.

@adrinjalali thanks, sounds great, will setup the base ideas in a PR within the coming week :)