58

I'm completely new to MongoDB & Mongoose and can't seem to find an answer as to how to handle migrations when a schema changes.

I'm used to running migration SQL scripts that alter table structure and any underlying data that needs to be changed. This typically involves DB downtime.

How is this typically handled within MongoDB/Mongoose? Any gotcha's that I need to be aware of?

2
  • 1
    Hi did you resolve this issue. I have connected to MongoDB using mongoose and would not like to execute migrations.
    – Xdrone
    Nov 20, 2013 at 8:55
  • I posted a simple example where the logic works in a local / live environments.
    – Ryan Q
    Mar 17, 2014 at 0:04

3 Answers 3

35

In coming across this and reasonably understanding how migrations work on a relational database, MongoDB makes this a little simpler. I've come to 2 ways to break this down. The things to consider when dealing with data migrations in MongoDB (not all that uncommon from RDBs) are:

  • Ensuring local test environments do not break when a developer merges the latest from the project repository
  • Ensuring any data is correctly updated on the live version regardless if a user is logged in or out if authentication is used. (Of course if everyone is automatically logged out when an upgrade is made, then only worrying about when a user logs in is necessary).

1) If your change will log everyone out or application downtime is expected then the simple way to do this is have a migration script to connect to local or live MongoDB and upgrade the correct data. Example where a user's name is changed from a single string to an object with given and family name (very basic of course and would need to be put into a script to run for all developers):

Using the CLI:

mongod
use myDatabase
db.myUsers.find().forEach( function(user){
    var curName = user.name.split(' '); //need some more checks..

    user.name = {given: curName[0], family: curName[1]};
    db.myUsers.save( user );
})

2) You want the application to migrate the schemas up and down based on the application version they are running. This will obviously be less of a burden for a live server and not require down time due to only upgrading users when they use the upgraded / downgraded versions for the first time.

If your using middleware in Expressjs for Nodejs:

  • Set an app variable in your root app script via app.set('schemaVersion', 1) which will be used later to compare to the users schema version.
  • Now ensure all the user schemas have a schemaVersion property as well so we can detect a change between the application schema version and the current MongoDB schemas for THAT PARTICULAR USER only.
  • Next we need to create simple middleware to detect the config and user version

    app.use( function( req, res, next ){
      //If were not on an authenticated route
      if( ! req.user ){
        next();
        return;
      }
      //retrieving the user info will be server dependent
      if( req.user.schemaVersion === app.get('schemaVersion')){
        next();
        return;
      }
    
      //handle upgrade if user version is less than app version
    
      //handle downgrade if user version is greater than app version
    
      //save the user version to your session / auth token / MongoDB where necessary
    })
    

For the upgrade / downgrade I would make simple js files under a migrations directory with an upgrade / downgrade export functions that will accept the user model and run the migration changes on that particular user in the MongoDB. Lastly ensure the users version is updated in your MongoDB so they don't run the changes again unless they move to a different version again.

3
  • 5
    Does this mean that you might have a middleware function for each schema in MongoDB? What are the performance or other trade-offs with this approach?
    – Peter
    Jan 19, 2018 at 11:18
  • This answer is very specific to migrating just a user's model, but how would you migrate other models. For example, let's say you have a Match model that is shared between two users, or a PublicEvent model that is created by a user, but can be seen by anybody? What strategies are there for coordinating these "live migrations" across different models, some related to a specific user and some not? Aug 12, 2018 at 7:14
  • In addition, you may reference a user record through a populate() call when accessing another model. This answer doesn't account for cases where user A references a the data on populated user B before the user B logs in and their record User record is updated. So, the referencing code still has to support both the old and new version of the schema, somehow. This may be scattered throughout the code and thus be very challenging to manage. Would love to see a more complete answer (or blog post) on how people manage this. Oct 8, 2018 at 23:53
15

If you're used to SQL type migrations or Rails-like migrations then you'll find my cli tool migrate-mongoose the right fit for you.

It allows you to write migrations with an up and a down function and manages the state for you based on success and failure of your migrations.

It also supports ES6 if you're using ES 2015 syntax.

You get access to your mongoose models via the this object, making it easy to make the changes you need to your models and schemas.

9

There are 2 types of migrations:

  • Offline: Will require you to take your service down for maintenance, then iterate over the entire collection and make the changes you need.

  • Online: Does not require to take your service down for maintenance. When you read the document, you check its version, and run a version specific migration routine for each version between the old and the new. Then you load the resulting thing.

Not all services can afford an offline migration, I recommend the online approach.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.