0

I'm working on a multi-tenant application running on mongodb. Each tenant can create multiple applications. The schema for most of the collections reference other collections via ObjectIDs. I'm thinking of manually creating a shard key with every record insertion in the following format:

(v3 murmurhash of the record's ObjectId) + (app_id.toHexString())

Is this good enough to ensure that records for any particular application will likely end up on the same shard?

Also, what happens if a particular application grows super large compared to all others on the shard?

1 Answer 1

0

If you use a hash based shard key with the input constantly changing (ObjectID can generally be considered to be unique for each record), then you will get no locality of data on shards at all (except by coincidence), though it will give you great write throughput by randomly distributing writes across all shards. That's basically the trade off with this kind of approach, the same is true of the built in hash based sharding, those trade offs don't change just because it is a manual hash constructed of two fields.

Basically because MongoDB uses range based chunks to split up the data for a given shard key you will have sequential ranges of hashes used as chunks in this case. Assuming your hash is not buggy in some way, then the data in a single sequential range will basically be random. Hence, even within a single chunk you will have no data locality, let alone on a shard, it will be completely random (by design).

If you wanted to be able to have applications grouped together in ranges, and hence more likely to be on a particular shard then you would be better off to pre-pend the app_id to make it the leftmost field in a compound shard key. Something like sharding on the following would (based on the limited description) be a good start:

{app_id : 1, _id : 1}

Though the ObjectID is monotonically increasing (more discussion on that here) over time, if there are a decent number of application IDs and you are going to be doing any range based or targeted queries on the ObjectID, then it might still work well though. You may also want to have other fields included based on your query pattern.

Remember that whatever your most common query pattern is, you want to have the shard key (ideally) satisfy it if at all possible. It has to be indexed, it has be used by the mongos to decide to route the query (if not, then it is scatter/gather), so if you are going to constantly query on app_id and _id then the above shard key makes a lot of sense.

If you go with the manual hashed key approach not only will you have a random distribution, but unless you are going to be querying on that hash it's not going to be very useful.

5
  • Thanks for the detailed answer! Originally I meant that I would not be hashing the app_id, just appending it to the hash of the row's _id but point taken about not needing to generate my own hashes and letting mongo handle it instead.
    – novon
    Oct 19, 2014 at 22:46
  • A few more questions. 1) Would there be an issue if app_id was not populated for every record? Specifically I'm allowing the creation of tenant level documents which could be shared among multiple apps. 2) Is there a downside to also adding tenant_id to the shard key hash? For example: {tenant_id: 1, app_id:1, _id: 1}? The goal of this would be to localize the a tenants documents, even if they do not belong to an app_id.
    – novon
    Oct 19, 2014 at 22:46
  • It could be a valid approach - you would probably not want to make it a sparse index though (just in case you were thinking of doing so). The issue there is that all the non-populated fields will be grouped together on a single shard, so you might have a hotspot if there was a lot of traffic without app_id. The combination of the two ID's, especially if the leftmost key is always populated, varies a bit for the non-populated app_id and is included in queries will be a better choice. As always, if possible, testing would be a good idea to verify. Oct 19, 2014 at 23:09
  • I see, for the sake of discussion lets assume app_id will always be populated. Since both app_id and _id are ObjectIds, which increase monotonically, this strategy will result in poor write distribution among the shards. Is it possible to pick a shard key that groups related app_ids together but also randomizes writes among a subset of the shards. Almost like loose localization where a particular app_id's records will be split among several (but not all) shards randomly?
    – novon
    Oct 20, 2014 at 4:02
  • One thing I would look at (depending on the uniqueness/timestamp requirements of course) would be how your ObjectIDs are being generated. The reason why they make for poor write scaling shard keys is that the most significant bits are a timestamp and that automatically increases over time when auto-generated. But, you could generate manually and avoid that problem. If that's not an option, then a compound key with some coarseness is what you are looking for - is there a field that satisfies that sort of loose selectivity at present? If not, then you might need to create yourself Oct 20, 2014 at 11:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.