73

If I delete the 3.1G journal file, sudo service mongodb restart will fail. However, this file is taking too much space. How can I solve this problem? How can I remove it?

bash$ du -sh /var/lib/mongodb/*
4.0K    _tmp
65M auction_development.0
128M    auction_development.1
17M auction_development.ns
3.1G    journal
4.0K    mongod.lock
0

3 Answers 3

76

TL;DR: You have two options. Use the --smallfiles startup option when starting MongoDB to limit the size of the journal files to 128MB, or turn off journalling using the --nojournal option. Using --nojournal in production is usually a bad idea, and it often makes sense to use different write concerns also in development so you don't have different code in dev and prod.

The long answer: No, deleting the journal file isn't safe. The idea of journalling is this:

A write comes in. Now, to make that write persistent (and the database durable), the write must somehow go to the disk.

Unfortunately, writes to the disk take eons compared to writes to the RAM, so the database is in a dilemma: not writing to the disk is risky, because an unexpected shutdown would cause data loss. But writing to the disk for every single write operation will decrease the database's performance so badly that it becomes unusable for practical purposes.

Now instead of writing to the data files themselves, and instead of doing it for every request, the database will simply append to a journal file where it stores all the operations that haven't been committed to the actual data files yet. This is a lot faster, because the file is already 'hot' since it's read and written to all the time, and it's only one file, not a bunch of files, and lastly, because it writes all pending operations in a batch every 100ms by default. Deleting this file in the middle of something wreaks havoc.

4
  • Information answer - is it possible to clear the journal files though? To free up space?
    – wmock
    Jun 24, 2014 at 10:17
  • 2
    @wmock Only if you are adding the smallfiles option in (Mitjas answer)[stackoverflow.com/a/25516871/1318694] and have cleanly stopped the database before removal. The journal files will be recreated if missing every time the database starts up. The smallfiles option changes how large those files are.
    – Matt
    Feb 11, 2015 at 8:25
  • I just wonder what's the rationale for having a 3 gig journal file by default. Do they expect a usual user will modify 3 gigs of data per 100 ms? Oct 8, 2015 at 10:03
  • Journal files are append only, so they are not deleted or emptied every 100ms. They are rotated once they reach 1gig of size, and will be deleted once they are no longer required (ie, when all changes have been applied). That introduces another level of complexity where the durability thread needs to make sure there is no pending write, so it's possible for two to three files to coexist. See docs.mongodb.org/manual/core/journaling/#journal-files
    – mnemosyn
    Oct 8, 2015 at 10:45
75

As explained in mnemosyn's answer, journaling is essential to the storage engines. Luckily, it can be controlled to some extent. The following was written for the MMAPv1 storage engine, which was the default until MongoDB 3.2. Then, WiredTiger became the engine of choice, to which more information can be found at the bottom of this answer.

MMAPv1

MongoDB < 2.6 (Non-YAML Config)

For our development server, we used the following procedure:

cp -p /etc/mongodb.conf /etc/mongodb.conf.orig
vi /etc/mongodb.conf

Now, insert

smallfiles=true

into the mongodb.conf, then save. smallfiles limits the journal file to 128MB.

service mongodb stop
rm -rf /var/lib/mongodb/journal/*
service mongodb start

MongoDB >= 2.6 (YAML Config)

If you're using MMAPv1 with the YAML config style, use the same step to backup the config as above, but into the

  mmapv1:

config block, insert

    smallFiles: true 

. Afterwards, proceed as above, restarting the server whilst removing the journals.

WiredTiger (MongoDB >=3.0, default since 3.2)

On development machines, journal files under WiredTiger should be somewhat smaller by default than under MMAPv1, as journal compression is enabled by default. According to the documentation, "WiredTiger journal files for MongoDB have a maximum size limit of approximately 100 MB". It will "create checkpoints (i.e. write the snapshot data to disk) at intervals of 60 seconds or 2 gigabytes of journal data."

Thus, if you're only running a low amount of requests (with little data to change) on your database, the journal files using WiredTiger should not exceed a low multiple of 100 MB. The size of the journal files seems not to be configurable, however.

4
  • 1
    The configuration file format changed in version 2.6 to YAML. See: docs.mongodb.org/manual/reference/configuration-options/…
    – Skarllot
    Jan 19, 2016 at 18:19
  • Thanks @Skarllot for pointing this out. As I'm no longer into MongoDB, I would be grateful if someone extended my answer by a section for version 2.6.
    – Mitja
    Jan 20, 2016 at 15:15
  • I updated the answer to reflect YAML and WiredTiger.
    – Mitja
    Oct 10, 2017 at 6:50
  • Is there a way to limit the number of journal files if we have deployed a RAID 10 cluster with limited space allocated for journal files ? Apr 3, 2018 at 20:31
10

mongodb has evolved since. Now its v3.4.1 stable.
I am on v3.2 here's how:
uncomment # mmapv1: so it looks like:

  mmapv1:
    smallFiles: true 

if you have a different version look for storage Options on the reference/configuration-options page.

don't forget to empty the journal

sudo service mongodb stop
sudo rm -rf /var/lib/mongodb/journal/*
sudo service mongodb start

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.