5

I'd like to programmatically make a snapshot of a live filesystem in Linux, preferably using LVM. I'd like not to unmount it because I've got lots of files opened (my most common scenario is that I've got a busy desktop with lots of programs).

I understand that because of kernel buffers and general filesystem activity, data on disk might be in some more-or-less undefined state.

Is there any way to "atomically" unmount an FS, make an LVM snapshot and mount it back? It will be ok if the OS will block all activity for few seconds to do this task. Or maybe some kind of atomic "sync+snapshot"? Kernel call?

I don't know if it is even possible...

  • 1
    I was on the LVM mailing list for awhile, and I seem to remember that filesystems are supposed to have hooks to be called to force the filesystem to checkpoint itself (i.e. set it up so that all journal entries are played for a journaled fs) just before an LVM snapshot operation. This would make sure the filesystem itself at least was in a consistent state in the snapshot. – Omnifarious Dec 21 '09 at 14:03
12

You shouldn't have to do anything for most Linux filesystems. It should just work without any effort at all on your part. The snapshot command itself hunts down mounted filesystems using the volume being snapshotted and calls a special hook that checkpoints them in a consistent, mountable state and does the snapshot atomically.

Older versions of LVM came with a set of VFS lock patches that would patch various filesystems so that they could be checkpointed for a snapshot. But with new kernels that should already be built into most Linux filesystems.

This intro on snapshots claims as much.

And a little more research reveals that for kernels in the 2.6 series the ext series of filesystems should all support this. ReiserFS probably also. And if I know the btrfs people, that one probably does as well.

| improve this answer | |
  • Of course, if you are making LVM snapshots of a btrfs partion, you're being rather silly. Btrfs handles all that better than LVM could. And btrfs doesn't need any checkpointing hook because of how it works. Either you get the superblock pointing at the new root or you don't. Of course, it probably does have such a hook, but it's almost superfluous. – Omnifarious Oct 27 '12 at 20:37
4

I know that ext3 and ext4 in RedHat Enterprise, Fedora and CentOS automatically checkpoint when a LVM snapshot is created. That means there is never any problem mounting the snapshot because it is always clean.

I believe XFS has the same support. I am not sure about other filesystems.

| improve this answer | |
  • 1
    "RedHat Enterprise, Fedora and CentOS automatically checkpoint when a LVM snapshot is created" : any documentation to prove this ? Is that specific to RH ? – Benoît Dec 4 '11 at 16:44
  • @Benoît: "many of the filesystems in the 2.6 kernel do this automatically when a snapshot is created" -- tldp.org/HOWTO/LVM-HOWTO/snapshotintro.html – Zan Lynx Dec 5 '11 at 16:47
  • 1
    Well, this is actually quite new kernelnewbies.org/… ! – Benoît Dec 8 '11 at 10:57
  • "RedHat Enterprise, Fedora and CentOS" - and which version of those would that be? – 0x89 Mar 25 '14 at 15:18
  • @0x89: Today in 2014 it is supported on everything recent. So use a modern Linux distro. – Zan Lynx Mar 25 '14 at 20:23
3

It depends on the filesystem you are using. With XFS you can use xfs_freeze -f to sync and freeze the FS, and xfs_freeze -u to activate it again, so you can create your snapshot from the frozen volume, which should be a save state.

| improve this answer | |
1

I'm not sure if this will do the trick for you, but you can remount a file system as read-only. mount -o remount,ro /lvm (or something similar) will do the trick. After you are done your snapshot, you can remount read-write using mount -o remount,rw /lvm.

| improve this answer | |
  • This is possible only when no file is opened for writing... with a desktop, lots of files are, even though apps aren't writing to them all the time; f.e. sqlite databases of firefox and other apps. – liori Dec 21 '09 at 13:48
1

Is there any way to "atomically" unmount an FS, make an LVM snapshot and mount it back?

It is possible to snapshot a mounted filesystem, even when the filesystem is not on an LVM volume. If the filesystem is on LVM, or it has built-in snapshot facilities (e.g. btrfs or ZFS), then use those instead.

The below instructions are fairly low-level, but they can be useful if you want the ability to snapshot a filesystem that is not on an LVM volume, and can't move it to a new LVM volume. Still, they're not for the faint-hearted: if you make a mistake, you may corrupt your filesystem. Make sure to consult the official documentation and dmsetup man page, triple-check the commands you're running, and have backups!

The Linux kernel has an awesome facility called the Device Mapper, which can do nice things such as create block devices that are "views" of other block devices, and of course snapshots. It is also what LVM uses under the hood to do the heavy lifting.

In the below examples I'll assume you want to snapshot /home, which is an ext4 filesystem located on /dev/sda2.

First, find the name of the device mapper device that the partition is mounted on:

# mount | grep home
/dev/mapper/home on /home type ext4 (rw,relatime,data=ordered)

Here, the device mapper device name is home. If the path to the block device does not start with /dev/mapper/, then you will need to create a device mapper device, and remount the filesystem to use that device instead of the HDD partition. You'll only need to do this once.

# dmsetup create home --table "0 $(blockdev --getsz /dev/sda2) linear /dev/sda2 0"
# umount /home
# mount -t ext4 /dev/mapper/home /home

Next, get the block device's device mapper table:

# dmsetup table home
home: 0 3864024960 linear 9:2 0

Your numbers will probably be different. The device target should be linear; if yours isn't, you may need to take special considerations. If the last number (start offset) is not 0, you will need to create an intermediate block device (with the same table as the current one) and use that as the base instead of /dev/sda2.

In the above example, home is using a single-entry table with the linear target. You will need to replace this table with a new one, which uses the snapshot target.

Device mapper provides three targets for snapshotting:

  • The snapshot target, which saves writes to the specified COW device. (Note that even though it's called a snapshot, the terminology is misleading, as the snapshot will be writable, but the underlying device will remain unchanged.)

  • The snapshot-origin target, which sends writes to the underlying device, but also sends the old data that the writes overwrote to the specified COW device.

Typically, you would make home a snapshot-origin target, then create some snapshot targets on top of it. This is what LVM does. However, a simpler method would be to simply create a snapshot target directly, which is what I'll show below.

Regardless of the method you choose, you must not write to the underlying device (/dev/sda2), or the snapshots will see a corrupted view of the filesystem. So, as a precaution, you should mark the underlying block device as read-only:

# blockdev --setro /dev/sda2

This won't affect device-mapper devices backed by it, so if you've already re-mounted /home on /dev/mapper/home, it should not have a noticeable effect.

Next, you will need to prepare the COW device, which will store changes since the snapshot was made. This has to be a block device, but can be backed by a sparse file. If you want to use a sparse file of e.g. 32GB:

# dd if=/dev/zero bs=1M count=0 seek=32768 of=/home_cow
# losetup --find --show /home_cow
/dev/loop0

Obviously, the sparse file shouldn't be on the filesystem you're snapshotting :)

Now you can reload the device's table and turn it into a snapshot device:

# dmsetup suspend home && \
  dmsetup reload home --table \
    "0 $(blockdev --getsz /dev/sda2) snapshot /dev/sda2 /dev/loop0 PO 8" && \
  dmsetup resume home

If that succeeds, new writes to /home should now be recorded in the /home_cow file, instead of being written to /dev/sda2. Make sure to monitor the size of the COW file, as well as the free space on the filesystem it's on, to avoid running out of COW space.

Once you no longer need the snapshot, you can merge it (to permanently commit the changes in the COW file to the underlying device), or discard it.

  • To merge it:

    1. replace the table with a snapshot-merge target instead of a snapshot target:

      # dmsetup suspend home && \
        dmsetup reload home --table \
          "0 $(blockdev --getsz /dev/sda2) snapshot-merge /dev/sda2 /dev/loop0 P 8" && \
        dmsetup resume home
      
    2. Next, monitor the status of the merge until all non-metadata blocks are merged:

      # watch dmsetup status home
      ...
      0 3864024960 snapshot-merge 281688/2097152 1104
      

      Note the 3 numbers at the end (X/Y Z). The merge is complete when X = Z.

    3. Next, replace the table with a linear target again:

      # dmsetup suspend home && \
        dmsetup reload home --table \
          "0 $(blockdev --getsz /dev/sda2) linear /dev/sda2 0" && \
        dmsetup resume home
      
    4. Now you can dismantle the loop device:

      # losetup -d /dev/loop0
      
    5. Finally, you can delete the COW file.

      # rm /home_cow
      
  • To discard the snapshot, unmount /home, follow steps 3-5 above, and remount /home. Although Device Mapper will allow you to do this without unmounting /home, it doesn't make sense (since the running programs' state in memory won't correspond to the filesystem state any more), and it will likely corrupt your filesystem.

| improve this answer | |
  • When I read "even when the filesystem is not on an LVM volume", I thought "this will be some good devicemapper black magic!" Magnificent! One question though—will this procedure trigger the same hooks that will ensure the snapshot is in a consistent state? I assume it would have to be part of the blockdev --setro /dev/sda2 call. – liori Feb 9 '17 at 20:53
  • You could add a sync call just before creating the snapshot, however since I can't find any reliable source on whether sync acts above or below the filesystem layer, that may or may not do anything. Regardless of that, the snapshot will be consistent in the meaning that all changes that will or will not be included in the snapshot will be delimited strictly by a point in time, and not e.g. disk offset or file name, so the snapshot will appear as if the machine had its power cut at that moment, as it should. [...] – Vladimir Panteleev Feb 9 '17 at 22:55
  • [...] This is sufficient e.g. for ACID databases to continue maintaining their data integrity. And unless you're using a non-journaling filesystem (ext3 and newer are journaling), I think filesystem corruption is also highly unlikely regardless whether filesystem caches have been flushed. – Vladimir Panteleev Feb 9 '17 at 22:55
  • Seems like it doesn't, though. LVM does it here: lxr.free-electrons.com/source/drivers/md/lvm.c?v=2.4.37#L2251. blockdev --setro eventually calls lxr.free-electrons.com/source/block/ioctl.c#L432, and it seems like just adding a call similar to the one from LVM would do the trick. – liori Feb 9 '17 at 23:32
0

FS corruption is "highly unlikely", as long as you never work in any kind of professional environment. otherwise you'll meet reality, and you might try blaming "bit rot" or "hardware" or whatever, but it all comes down to having been irresponsible. freeze/thaw (as mentioned a few times, and only if called properly) is sufficient outside of database environments. for databases, you still won't have a transaction-complete backup and if you think a backup that rolls back some transaction is fine when restored: see starting sentence. depending on the activity you might just added another 5-10 mins of downtime if ever you need that backup. Most of us can easily afford that, but it can not be general advice. Be honest about downsides, guys.

| improve this answer | |
  • Thank you for your note. However, please do not add comments to other answers as separate answers. That's what comments are for in the StackExchange platform. I suggest you add the part about file system corruption as a comment to Vladimir Panteleev's answer (which I assume you refer to), and the part about databases would be better placed as a comment to the question itself. – liori Mar 19 '17 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.