0

The problem I'm trying to solve is finding the right similarity metric, rescorer heuristic and filtration level for my data. (I'm using 'filtration level' to mean the amount of ratings that a user or item must have associated with it to make it into the production database).

Setup
I'm using mahout's taste collaborative filtering framework. My data comes in the form of triplets where an item's rating are contained in the set {1,2,3,4,5}. I'm using an itemBased recommender atop a logLikelihood similarity metric. I filter out users who rate fewer than 20 items from the production dataset. RMSE looks good (1.17ish) and there is no data capping going on, but there is an odd behavior that is undesireable and borders on error-like.

Question

First Call -- Generate a 'top items' list with no info from the user. To do this I use, what I call, a Centered Sum:

for i in items
 for r in i's ratings
  sum += r - center

where center = (5+1)/2 , if you allow ratings in the scale of 1 to 5 for example

I use a centered sum instead of average ratings to generate a top items list mainly because I want the number of ratings that an item has received to factor into the ranking.

Second Call -- I ask for 9 similar items to each of the top items returned in the first call. For each top item I asked for similar items for, 7 out of 9 of the similar items returned are the same (as the similar items set returned for the other top items)!

Is it about time to try some rescoring? Maybe multiplying the similarity of two games by (number of co-rated items)/x, where x is tuned (around 50 or something to begin with).

Thanks in advance fellas

1 Answer 1

0

You are asking for 50 items similar to some item X. Then you look for 9 similar items for each of those 50. And most of them are the same. Why is that surprising? Similar items ought to be similar to the same other items.

What's a "centered" sum? ranking by sum rather than average still gives you a relatively similar output if the number of items in the sum for each calculation is roughly similar.

What problem are you trying to solve? Because none of this seems to have a bearing on the recommender system you describe that you're using and works. Log-likelihood similarity is not even based on ratings.

3
  • Thanks for your quick reply Sean! The problem I'm trying to solve is finding the right similarity metric and rescorer heuristic for my data. Sorry for the ambiguity, I've updated my question with a description of what I'm calling a "centered" sum. Your feedback about Log-Likelihood is most appreciated. I switched to using LL because GenericItemRecommender must cap less values when used with LL relative to Pearson (since negative weights are allowed). Maybe I should switch back to using Pearson, implementing some form of rescorer this time? Aug 16, 2011 at 19:07
  • Your first process is mostly constructing a list of most-rated items. Is your data set small or sparse? If so, I can imagine that it's only the most-rated items that tend to have any defined similarity to other items, because of sparseness. That could explain why you see the same items over and over. This still isn't making use of the Recommender -- unless you're just using mostSimilarItems().
    – Sean Owen
    Aug 16, 2011 at 21:44
  • Thanks for the help Sean. The problem as it turned out was that I was sorting similar items by gameID instead of similarity value! Wups. Also, switching from Log-likelihood to Pearson since I have rating data was a good call I think (although capping still kind of scares me). Thanks for your assistance Sean! Aug 18, 2011 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.