41

I want to add the unique index to a field ignoring null values in the unique indexed field and ignoring the documents that are filtered based on partialFilterExpression.

The problem is Sparse indexes can't be used with the Partial index.

Also, adding unique indexes, adds the null value to the index key field and hence the documents can't be ignored based on $exist criteria in the PartialFilterExpression.

Is it possible in MongoDB 3.2 to get around this situation?

2
  • Does the null value have any special meaning in your case? If not, you can always $unset the field for all null values which makes it possible to use the $exist operator.
    – joao
    Commented Mar 2, 2016 at 19:23
  • 1
    Adding unique index auto adds the null field so we can't have documents without having unique indexed key field.
    – Nikhil
    Commented Mar 3, 2016 at 5:39

7 Answers 7

66

I am adding this answer as I was looking for a solution and didn't find one. This may not answer exactly this question or may be, but will help lot of others out there like me.

Example. If the field with null is houseName and it is of type string, the solution can be like this

db.collectionName.createIndex(
   {name: 1, houseName: 1},
   {unique: true, partialFilterExpression: {houseName: {$type: "string"}}}
);

This will ignore the null values in the field houseName and still be unique.

6
  • 11
    Just a note from my testing for anyone else looking for the same solution: although this worked as a mechanism for enforcing uniqueness while allowing nulls, Mongo didn't use the index for lookup queries using the indexed field and using hint() to force it resulted in very slow performance
    – Pete S
    Commented Apr 20, 2018 at 10:25
  • @PeteS I saw the same behaviour, did you find a solution that works? Commented May 31, 2018 at 14:11
  • 3
    @MattOakley Not exactly. We went with a workaround solution of setting partialFilterExpression: { fieldToIndex: { $exists: true } } on the index then decorating the corresponding property in our class in the app code with [BsonIgnoreIfNull] (C#). The Mongo driver will then not serialize the field if it's null in the app when saving.
    – Pete S
    Commented Jun 1, 2018 at 15:34
  • 3
    If partial index contains $type: string you should add this to your query to use index (like .find({ houseName: { $type: "string", $eq: "some value" } }))
    – oryol
    Commented Feb 4, 2019 at 8:01
  • @PeteS and @oryol are corrects, using {$type: "string"} in default requests, not improves performance, must use { $exists: true }
    – Armando
    Commented Mar 14, 2020 at 16:45
11

Here is an example that I modified from the mongoDB partial index documentation:

db.contacts.createIndex(
   { email: 1 },
   { unique: true, partialFilterExpression: { email: { $exists: true } } }
)

IMPORTANT

To use the partial index, a query must contain the filter expression (or a modified filter expression that specifies a subset of the filter expression) as part of its query condition.

You can see that queries such as:

db.contacts.find({'email':'[email protected]'}).explain()

will indicate that they doing an index scan, even if you don't specify {$exists: true} because you're implicitly specifying a subset of the partialFilterExpression by specifying an email in your filter.

On the other hand, the following query will do a collection scan:

db.contacts.find({email: {$exists: false}})

WARNING

mythicalcoder's answer (currently the highest voted answer) is very misleading because it successfully creates a unique index, but the query planner will not generally be able to use the index you've created unless you add houseName: {$type: "string"} into your filter expression. This can have performance costs which you might not be aware of and can cause problems down the road.

3
  • 1
    This seems to be the best answer 2022+.
    – syvex
    Commented Jun 2, 2023 at 15:13
  • $exists doesn't work with null values so it's better to use $type: "string" instead Commented Jan 5 at 20:43
  • mongodb.com/docs/manual/reference/operator/query/exists/…. "The $exists operator matches documents that contain or do not contain a specified field, including documents where the field value is null."
    – jacob
    Commented Jan 8 at 21:05
8

You can create partial index in mongo:3.2. Example, if ipaddress can be "", but "127.0.0.1" should be unique. The solution can be like this:

db.collectionName.createIndex(
 {"ipaddress":1},
 {"unique":true, "partialIndexExpression":{"ipaddress":{"$gt":""}}})

This will ignore "" values in ipaddress filed and still be unique

7

Yes, you can create partial index in MongoDB 3.2

Please see https://docs.mongodb.org/manual/core/index-partial/#index-type-partial

MongoDB recommend usage of partial index over sparse index. I'll suggest you to drop your sparse index in favor of partial index.

2
  • I have to use PartialFilterExpression to filter data for indexing and on the side have to use the unique index.
    – Nikhil
    Commented Mar 3, 2016 at 5:39
  • The only thing is I want some documents to not have the unique indexed key field which seems not possible or I may not aware of.
    – Nikhil
    Commented Mar 3, 2016 at 5:48
4

{ "YourField" : { "$exists" : true, "$gt" : "0", "$type" : "string" } }

1
  • 1
    This answer is perfect if you want an index that is unique for all values that are not equal to an empty string or null. Commented Jan 19, 2021 at 21:22
0

To create at mongodbCompass you must write it as JSON:

for find other types wich supports see this link.

enter image description here

0

Yes, that can be a kind of a problem that the partial filter expression cannot contain any 'not' filters.

For those who can be interested in a C# solution for an index like this, here is an example.

We have a 'User' entity, which has one-to-one 'relation' to a 'Doctor' entity. This relation is represented by the not required, nullable field 'DoctorId' in the 'User' entity. In other words, there is a requirement that a given 'Doctor' can be linked to only single 'User' at a time.

So we need an unique index which can fire an exception when something attempts to set DoctorId to the same Guid which already set for any other 'User' entity. At the same time multiple 'null' entries must be allowed for the 'DoctorId' field, since many users do not have any doctor attached to them.

The solution to build this kind of an index looks like:

var uniqueDoctorIdIndexDefinition = new IndexKeysDefinitionBuilder<User>()
    .Ascending(o => o.DoctorId);
var existsFilter = Builders<User>.Filter.Exists(o => o.DoctorId);
var notNullFilter = Builders<User>.Filter.Type(o => o.DoctorId, BsonType.String);
var andFilter = Builders<User>.Filter.And(existsFilter, notNullFilter);
var createIndexOptions = new CreateIndexOptions<User>
{
    Unique = true,
    Name = UniqueDoctorIdIndexName,
    PartialFilterExpression = andFilter,
};
var uniqueDoctorIdIndex = new CreateIndexModel<User>(
    uniqueDoctorIdIndexDefinition,
    createIndexOptions);

users.Indexes.CreateOne(uniqueDoctorIdIndex);

Probably in your description of a 'User' entity you must directly specify the BsonType of the 'DoctorId' field, by using an attribute, for example in our case it was:

[BsonRepresentation(BsonType.String)]
public Guid? DoctorId { get; set; }

I am more than sure that there is a more proficient and compact solution for this problem, so would be happy if somebody suggests it here.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.