1

So I have an interesting use case that I'm stuck trying to find a efficient mongo query for.

To begin, I have 12,000 categories with 100,000 posts. I need to randomly select a 100 pairs of posts, from random categories. The pairs are randomly selected from categories, but each pair must have both posts belonging to the same category.

Users look at each pair to rate and once they finish looking at the 100, they fetch another 100 random posts (preferably not any of the same pairs they've already seen).

So the requirements are:

  • Fetch 100 pairs of posts randomly from a random set of categories

Optional requirements:

  • Not to return the same pairs they've already rated

Mongo Collections

  • Users
  • Categories
  • Posts
    • CategoryId
    • Ratings (embedded collection in posts)

How would I do this in Mongo... should I move some of this data off of mongo to another db if it's easier?

0

2 Answers 2

0

Yes. Very interesting question. My suggestion is to put a randomVal field on your post documents. Then you can sort on {CategoryId: 1, randomVal: 1}. The result will be a cursor that groups all the posts by CategoryId but randomly within that grouping. If you conceptually think of this as an array, you can pick all the even indexed posts, and pair them with an odd neighbor to get unique random pairs within categories.

I think that how to select the random pairs from this list will take some experimentation, but my gut instinct is that the best approach would be to have a separate process that periodically caches a collection of pairs which are sorted by a separate randomVal2. The user facing queries would just increment through this pairs collection 100 at a time.

0

I think you can achieve this in two query. First you need to use aggregation framework and do a map reduce operation on Posts collection. In the map phase use category id as the key and emit post ids to reducer.

In the reduce phase choose two random id from each category. In the end of the map reduce you will have a list of Post ids. Then retrieve those posts from Posts collection.

Add a ratedBy field to Post document and when user rated a post add his or her userName to ratedBy field. Then use that field as a filter to your map reduce command in the first place so that you don't bring already rated documents to user.

Good luck

2
  • How would I also do it randomly? I wouldn't necessarily want to scan all posts or categories so is there a way to just grab a random set? Jan 23, 2013 at 16:24
  • Whats the structure of your category and post, how do you identify a category or post, what is the type of their ids?
    – cubbuk
    Jan 23, 2013 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.