2

The word on the street is that MongoDB gets slow if you can't keep the indexes you're using in memory. How does this work with sharding? Does a sharded only keep its own BTree in memory, or does every shard need to keep the index for the entire collection in memory?

2 Answers 2

7

Does a sharded only keep its own BTree in memory...?

Yes, each shard manages its own indexes.

The word on the street is that MongoDB gets slow if you can't keep the indexes you're using in memory.

You can actually expect worse when using sharding and secondary indexes. The key problem is that the router process (mongos) knows nothing about data in secondary indexes.

If you do a query using the shard key, it will be routed directly to the correct server(s). In most cases, this levels out the workload. So 100 queries can be spread across 100 servers and each server only answers 1 query.

However, if you do a query using the secondary key, that query has to go to every server. So 100 queries to the router will result 10,000 queries across 100 servers or 100 queries per server. As you add more servers, these "non-shardkey" queries become less and less efficient. The workload does not become more balanced.

Some details are available in the MongoDB docs here.

2
  • if you are sharding on more than one key, can you still make efficient queries on all of those keys?
    – nnythm
    Jan 31, 2012 at 20:48
  • 1
    Yes, but subject to the limitations on indexing and sorting on compound keys. You'll have to test this and know which queries will work, the order of the keys is very important.
    – Gates VP
    Jan 31, 2012 at 22:43
0

Just its own portion of the index (it doesn't know about the other shards' data). Scaling wouldn't work very well, otherwise. See this documentation for some more information about sharding: http://www.mongodb.org/display/DOCS/Sharding+Introduction

http://www.mongodb.org/display/DOCS/Choosing+a+Shard+Key

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.