4

I have a slow query in my application. After created two indexes it uses them with better performance in local DB. But when I deployed on production DB it still uses the origin index.

Below this what I did.

Properties in collection tasks: team_id, project_id, created_by and assignee, etc.

The query looks like below

db.tasks.find({
  team_id: new ObjectId(teamId),
  $or: [
    {
      project_id: newObjectId(projectId),
      created_by: userId
    },
    {
      assignee: userId
    }
  ]
})

Originally there's only one index against team_id, which will check over 10k docs. Then I added two new indexes

project_1_created_by_1: {
  project: 1,
  created_by: 1
}

assignee_1: {
  assignee: 1
}

In local DB I ran my query with explain({ verbose: true }). I can see MongoDB evaluated indexes

[
  QueryOptimizerCursor: [
    'project_1_created_by_1',
    'assignee_1',
  ],
  BtreeCursor: 'team_1'
]

Finally QueryOptimizerCursor won.

But when I ran it on production MongoDB the result of explain({ verbose: true }) shown it only evaluated team_1 and BasicCursor.

[
  BtreeCursor: `team_1`,
  BasicCursor
]

Does anyone give me some information why MongoDB didn't use new indexes I created, even worse it didn't evaluate it.

PS: I can confirm the new indexes were ready in my production database since when I use query db.tasks.find({project: xxx, created_by:yyy}).explain() it uses the new one I created.

Updated

The version of of production MongoDB was 2.4.12 while local was 2.6.7. When I installed a new copy of MongoDB 2.4.12 at local and ran the same query it used team index rather than QueryOptimizerCursor.

Not pretty sure if this is only because of MongoDB 2.6.7 is more smarter than 2.4.12.

1 Answer 1

5

If a query can be satisfied by multiple indexes defined in the collection, MongoDB will test all the applicable indexes in parallel. The first index that can return 101 results will be selected by the query planner. There are other facets to index selection, but in general this is true as per the Query Optimization documentation.

This index selection method may select a sub-optimal index. This is because to MongoDB's point of view, you have multiple indexes that describe the same thing. To mitigate the sub-optimal index selection you observed, you can do:

  1. Remove all other indexes that you discover to be sub-optimal.

    This is to ensure that the query planner has no other choice except to choose indexes that you tailored for your query.

  2. Use the hint() method

    hint() allows you to explicitly tell MongoDB to use the prescribed index for the query. For example:

    db.tasks.find(...).hint({project: 1, created_by: 1})
    

    Please see https://docs.mongodb.com/v2.6/reference/operator/meta/hint/ for more information regarding hint().

Another nuance in your query is that it includes an $or operator. In this case, every term in the $or expression must have an index associated with it, otherwise MongoDB will perform a collection scan (BasicCursor in MongoDB 2.6 terms). This is explained in more details at https://docs.mongodb.com/v2.6/reference/operator/query/or/#behaviors

2
  • Thanks for your answer. I think I have to remove indexes and discover the best one. Just one more question, seems that I can only specify one index through hint(), but cannot multiple indexes used by my $or operator.
    – Shaun Xu
    Jul 18, 2016 at 7:01
  • hint() only allows one index. In most cases, it's better to remove unnecessary indexes since too many indexes could slow down your insert, since every insert that touch an index term would mean that the corresponding index must be updated as well.
    – kevinadi
    Jul 18, 2016 at 7:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.