I have a Mongo database that I did not create or architect, is there a good way to introspect the db or print out what the structure is to start to get a handle on what types of data are being stored, how the data types are nested, etc?
10 Answers
Just query the database by running the following commands in the mongo shell:
use mydb //this switches to the database you want to query
show collections //this command will list all collections in the database
db.collectionName.find().pretty() //this will show all documents in the database in a readable format; do the same for each collection in the database
You should then be able to examine the document structure.
-
5What if some documents are different to others? That method won't show ALL fields or the entire structure stored.– SammayeFeb 5, 2013 at 17:33
-
1@Sammaye you're right...it will need a script to reach into all docs and compare all keys then print a tree structure is probably a good visual way to show the whole structure and make it digestible– br3w5Feb 5, 2013 at 17:45
-
3
-
The script uses
typeof
function. It retrives js types and not BSON types. Mar 20, 2016 at 12:03
There is actually a tool to help you out here called Variety:
http://blog.mongodb.org/post/21923016898/meet-variety-a-schema-analyzer-for-mongodb
You can view the Github repo for it here: https://github.com/variety/variety
I should probably warn you that:
- It uses MR to accomplish its tasks
- It uses certain other queries that could bring a production set-up to a near halt in terms of performance.
As such I recommend you run this on a development server or a hidden node of a replica or something.
Depending on the size and depth of your documents it may take a very long time to understand the rough structure of your database through this but it will eventually give one.
-
6New URL, and it has been recently updated as of today 2014-10-14. github.com/variety/variety Variety, a Schema Analyzer for MongoDB. Very helpful! Oct 14, 2014 at 15:13
This will print name and its type
var schematodo = db.collection_name.findOne()
for (var key in schematodo) { print (key, typeof key) ; }
-
1For some reasons, for me it returns type "string" for every fields while there type ObjectId and Number exist. Apr 7, 2021 at 6:42
-
1
I would recommend limiting the result set rather than issuing an unrestricted find command.
use mydb
db.collectionName.find().limit(10)
var z = db.collectionName.find().limit(10)
Object.keys(z[0])
Object.keys(z[1])
This will help you being to understand your database structure or lack thereof.
-
The only problem is that there is no way the say that the number you get back represents the database structure, you could have 1000001 records and limit on 1000000 of them but miss the existing field in the last document because the other 1000000 dont have it– SammayeJul 28, 2014 at 12:41
-
I agree with you there. However for a quick and dirty it might suffice. I know of many use cases where mongo is used as a semi structured data store similar to traditional RDMS. In the other case this technique will not give the full structure but it can at least provide something quick. I have used the variety tool and like it a lot for a comprehensive schema analysis.– PaulJul 30, 2014 at 14:22
-
True and there is a tool out there somewhere, forgot its name, that actually takes 10 queries of limited 10 queries in an attempt to create a performant schema analyzer.– SammayeJul 30, 2014 at 14:23
This is an open-source tool that I, along with my friend, have created - https://pypi.python.org/pypi/mongoschema/
It is a Python library with a pretty simple usage. You can try it out (even contribute).
One option is to use the Mongoeye. It is open-source tool similar to the Variety.
The difference is that Mongoeye is a stand-alone program (Mongo Shell is not required) and has more features (histograms, most frequent values, etc.).
Few days ago I found GUI client MongoDB Compass with some nice visualizations. See the product overview. It comes directly from the mongodb people and according to their doc:
MongoDB Compass is designed to allow users to easily analyze and understand the contents of their data collections within MongoDB...
-
2
You may've asked about validation schema. Here's the answer how to get it: How to retrieve MongoDb collection validator rules?
Use Mongo Compass
which does a sample as explained here Which does a random sample of 1000 documents to get you the schema - it could miss something but it's the only rational option if you database is several GBs.
Visualisation
The schema then can be exported as JSON
You can use MongoDB's tool mongodump
. On running it, a dump
folder is created in the directory from which you executed mongodump. In that folder, there are multiple folders that correspond to the databases in MongDB, and there are subfolders that correspond to the collections, and files that correspond to the documents.
This method is the best I know of, as you can also make out the schema of empty collections.