4

I'm looking for a way to keep track of what blocks on a block device are modified after a point in time. How I eventually want to use this for is to keep two 2TB disks in sync, one which only comes online (connected through USB) once a month. Without knowing what blocks have been modified, I have to go through the whole 2TB every time.

I'm using a recent GNU/Linux OS and have C and Python experience. I'm hoping to avoid writing kernel level code as I don't have any experience in that area whatsoever. My current theory is that there should be some hooks somewhere where my code can get called when a disk flush is performed.

Any ideas?

0

3 Answers 3

4

It should be possible to use Linux MD for this, provided you're careful to avoid a bug in the block layer. Every month or so, you add the USB disk as a new member of a 2-disk RAID set where one is missing by default, and let it do the synchronization of changed blocks. A write-intent bitmap seems beneficial for that, so don't forget to have one around.

# Creation
mdadm -C /dev/md0 -l 1 -n 2 -e 1.0 -b internal  /dev/sda  missing

# Addition of slave disk
mdadm /dev/md0 -a /dev/thatusbthing

See also a longer description of this setup, with more discussion of options/potential pitfalls.

Addendum:

rsync was designed to transfer files over a (comparatively slow) network. That means both sides will scan their device locally, compute that rolling checksum, and then transfer the chunks that changed. The changelist is of course dependent on calculating the checksums. (Reading at 30+MB/s from a disk is faster than unconditionally pushing at, say, 10MB/s, over a 100mbit network.)

With MD write-intent bitmaps, the scan stage is not necessary, because it already knows, by means of this bitmap, which blocks have been changed since the disks were last synchronized.

4
  • I'm not sure how this would differ from an inplace rsync, except by taking place mostly in kernel space (and running lazily). Is there some additional data on RAID devices that records changes that I'm not aware of? Jan 7, 2011 at 2:21
  • I edited the question now under Addendum:, as the comment function does not have enough room.
    – user562374
    Jan 7, 2011 at 21:20
  • If this write intent bitmap stuff sticks around, this would be the most efficient way to do this by far, +1 Jan 13, 2011 at 3:45
  • Stick around? Of course it is written to disk, otherwise there would be no point in keeping track.
    – user562374
    Jan 14, 2011 at 1:36
2

You can use drbd. It looks like an bigga overhead but you can use two devices on one machine. The second device (that on usb) should be removed most of the time and set to state "secondary". After you plug it in it should synchronize very fast. This is because drbd keeps track of local changes.

Yes, there's a drawback: You have to use the special format, not just a raw device and the device will be smaller, because it needs that bitmap for locally-changed sectors.

1

Without making use of any contained filesystem metadata, I believe the only way to determine changes is by comparing the block devices. You may be able to apply the delta-transfer algorithm of rsync with rsync --inplace -B 4K /dev/sdX /dev/sdY. Adjust the block size as appropriate for your block devices. This should require 4TB of reads, but won't require that the block devices be on the same system. You can enable compression in the transfer among numerous other options.

1
  • Just a fyi, rsync skips non-regular files (so /dev/sdX does not work) and atleast with my version -B doesn't accept 4K but requires an actual number.
    – Wolph
    May 31, 2011 at 16:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.