5

I'm very new to couchdb and I'm wondering how I can create ID's that look like.

Employee:DBX-**0001**-SP

the number portion 0001 must be unique and sequential. How can I achieve something like this in couchdb? I searched all over and I cannot find any simple solution.

It would be best if I can generate the sequential portion in couchdb and not on the client side to avoid collisions during replication.

The current solution I have is that I fetch a document I've stored that looks like this {"_id": "EmployeeAutoIncrement", value: 1} upon retrieval I increment the value and send it back to the server if those are successful then I return the new incremented value and use it as my Auto Increment Value to be part of the ID Employee:DBX-AUTO_INCREMENT_VALUE_HERE-SP

The issue with this is that if two people make a request to the EmployeeAutoIncrement at the same time and they both update it will it not cause conflicts? Also, if one person makes a request and they go offline then they come back online then wouldn't that also make a conflict?

5
  • If I were you I would try to use UTC date via JSON.stringify(new Date()) as the auto increment. It's unique, sequential and as a bonus stores the date created. I'm not sure how this can be done on the server side so there could be an issue with multiple clients trying to create a document at the same millisecond. You could retry once if it fails due to the _id already existing and that should account for most cases. Aug 21, 2017 at 2:35
  • Sorry but I don't understand how I can get a sequence number from that. It should be in order like 0001 0002 0003 0004 0005. Please elaborate how JSON.stringify(new Date()) will give me 0001 0002 ... Aug 21, 2017 at 7:24
  • It won't give you a sequence without gaps, but it can easily be sorted in ascending order if that's all you need Aug 21, 2017 at 7:43
  • Yea that's what I thought. Good idea but... I do need a sequence without gaps because it's the requirement. BTW did you mean I have to use the unix timestamp as the sequence or the complete UTC date string? Aug 21, 2017 at 7:54
  • @ManiMuridi you are trying to violate the CAP theorem if you want to display the correct numbers and have partitioned clients.. You have to find out the priority of your requirements.
    – lossleader
    Aug 26, 2017 at 13:06

2 Answers 2

2

All of the requirements cannot be satisfied client-side when using multiple clients, some of which might be off-line.

Here is a process that results in a monotonically-increasing id:

  1. Each client saves a record with a unique id. The record should include a flag marking the record as temporary.
  2. Build an external process that listens to the changes feed for records marked as temporary. The changes feed outputs records in "time order of application".
  3. The external process should create a new record with the correct id, flagging it as permanent. Since only that process creates "permanent" records, it can read and write the EmployeeAutoIncrement value without collisions.
  4. The external process can then delete the temporary record.

The database will have double the number of records, so it will grow more quickly and need to be compacted sooner if space is an issue. Any views/queries on the employee records will need to check for the permanent flag, in case a query runs while the external process is adding a new record.

1
+50

It can (sort of) be done, although I recommend you think about your design choices and why you want this done on a distributed database -- it's probably better done on the client where you can control the serialization to your sequence generator.

If you want to do it at least partially on the server, you will need an implementation of a so-called CRDT counter, as outlined in the following paper:

http://hal.upmc.fr/docs/00/55/55/88/PDF/techreport.pdf

You can find a Ruby implementation of some of those ideas here:

https://github.com/aphyr/meangirls

and a simple Couch-specific implementation of a counter (the one you need) and a set here:

https://github.com/drsm79/couch-crdt

The latter, whilst written in Python, will do almost exactly what you want, if you follow the pattern as shown in the following example:

https://github.com/drsm79/couch-crdt/blob/master/examples/counter.py

which will give you your monotonic sequence. From there, create your document _id.

Translation to JavaScript and PouchDB left as an exercise for the reader.

1
  • Although I still don't have a good solid solution for this, you did provide a lot of helpful links which gave me ideas to try. Hence the reason I chose to give you the bounty. Thanks a lot! (y) Aug 27, 2017 at 8:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.