2

I have two folders FolderA and FolderB as below. I want to rsync the common subfolders. For example, I can do rsync -avzP /path/to/FolderB/* /path/to/FolderA/, which will keep SubFolder1 and SubFolder3 mirrored. My question is how I can achieve the same if FolderB is the destination without explicitly --include or --exclude individual subfolders (e.g., in case there are too many of them).

FolderA
|--SubFolder1
|--SubFolder2
|--SubFolder3
|--SubFolder4

FolderB
|--SubFolder1
|--SubFolder3

1 Answer 1

1

You can update all files in FolderA using the contents FolderB as the source with your normal:

rsync -uav /path/to/FolderB/ /path/to/FolderA

(note: the trailing '/' after FolderB/ is mandatory to copy the contents of FolderB rather than FolderB itself)

To do it in reverse and update FolderB from FolderA and not copy SubFolder2 and SubFolder4 with the --existing option which will "skip creating new files on receiver", but that will also prevent new files and directories within SubFolder1 and SubFolder3 from being created as well.

You best option to not copy SubFolder2 and SubFolder4 while allowing new files and directories within SubFolder1 and SubFolder3 be created is to use the --filter option. See rsync(1) - Linux manual page.

A typical way to use --filter to exclude SubFolder2 and SubFolder4 on a copy from FolderA to FolderB would be:

rsync -uav --filter -_SubFolder2/ --filter -_SubFolder4/ /path/to/FolderA/ /path/to/FolderB

That will allow you to copy the complete contents of /path/to/FolderA/ to /path/to/FolderB/ without including SubFolder2 and SubFolder4.


Edit Per-Comment On Large Number of SubFolders

If you have a large number of folders under FolderA that you do not want to sync under FolderB, then your other option is to create a text file holding the absolute path to only those SubFolderX under FolderA you want to rsync to FolderB and then use the --no-R and --files-from=folderlist options to only rsync the wanted SubFolders. This will eliminate having to specify a large number of --filter options on the command line.

For example, you can create your folderlist with:

find /path/to/FolderA -maxdepth 1 -type d > folderlist

(note: specify the absolute path above and find will produce the folderlist file containing absolute paths)

Now edit your folderlist file and remove the parent directory (e.g. /path/to/FolderA) and any SubFolders you don't want to sync under FolderB. You can now use the folderlist file to control which SubFolders under FolderA are sync'ed to FolderB without having to include a long list of filters on the command line. Your command line then becomes

rsync -uai -r --no-R --files-from=folderlist / /path/to/FolderB

(note: the '/' as source serves as a base for the paths contained in folderlist. You can change the -i option to control the level of information dumped to the screen, e.g. -v, etc... or remove it altogether to suppress any reporting other than errors)

(also note: when using --files-from, -a does not imply -r (recursive), so you will need to explicitly add -r if you need a recursive transfer)

7
  • My use case is that there are a lot of common folders to sync (SubFolder1, SubFolder2, ...) but there are also a lot of folders to skip (SubFolder2, SubFolder4, ...), so really hoping for a way without doing anything explicit. Also, wouldn't rsync -uav /path/to/FolderB/ /path/to/FolderA remove SubFolder2 and SubFolder4 under FolderA?
    – Roc W.
    Jul 30, 2019 at 12:34
  • You can always create a list of subfolders to copy in a text file, 1-per line containing the Absolute path, e.g. /path/to/FolderA/SubFolder1 /path/to/FolderA/SubFolder3 ... and then call rsync with the rsync -uai --no-R --files-from=thatfile / /path/to/FolderB. That way you only have to create the folder list once and then you can avoid any exclusions in the rsync call itself. Jul 30, 2019 at 17:59
  • Ah, that's a good idea! Unfortunately I don't have enough reputation to upvote yet..
    – Roc W.
    Jul 31, 2019 at 18:04
  • Will do. I do believe this is a much better solution than what I had before, but I just wanted to give the question a few more days of exposure to potentially more ideas.
    – Roc W.
    Aug 1, 2019 at 14:50
  • 1
    I was just implementing this today and it worked wonderfully. One small omission is that -r is needed in addition to -a when using --files-from.
    – Roc W.
    Aug 3, 2019 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.