88

I need to reinstall one of ours servers, and as a precaution, I want to move /home, /etc, /opt, and /Services to backup server.

However, I have a problem: because of plenty of symbolic links a lot of files are copied multiple times.

Is it possible to make scp ignore the symbolic links (or actually to copy link as a link not as a directory or file)? If not, is there another way to do it?

5 Answers 5

91

I knew that it was possible, I just took wrong tool. I did it with rsync

rsync --progress -avhe ssh /usr/local/  XXX.XXX.XXX.XXX:/BackUp/usr/local/
4
  • 2
    So what does this command? Does it ignore symlinks or create new ones? -avhe? Aug 10, 2016 at 13:20
  • 5
    a is archive (preserves things), v verbose, h human readable output, e specify the service you want to use (in this case it's ssh). Read more over here Aug 10, 2017 at 23:19
  • Thank you. Old server being mothballed. New server needed content. Lost all the soft-links but many were broken, so this will just now be a slow maintenance task on the new server. Thanks again.
    – jlettvin
    Aug 18, 2019 at 20:22
  • 2
    --safe-links allows to preserve links that are inside the directory and to not copy the ones that point outside of it. Oct 30, 2019 at 7:33
34

I found that the rsync method did not work for me, however I found an alternative that did work on this website (www.docstore.mik.ua/orelly).

Specifically section 7.5.3 of "O'Reilly: SSH: The Secure Shell. The Definitive Guide".

7.5.3. Recursive Copy of Directories

...

Although scp can copy directories, it isn't necessarily the best method. If your directory contains hard links or soft links, they won't be duplicated. Links are copied as plain files (the link targets), and worse, circular directory links cause scp1 to loop indefinitely. (scp2 detects symbolic links and copies their targets instead.) Other types of special files, such as named pipes, also aren't copied correctly.A better solution is to use tar, which handles special files correctly, and send it to the remote machine to be untarred, via SSH:

$ tar cf - /usr/local/bin | ssh server.example.com tar xf -
5
  • 2
    This is a good suggestion if rsync isn't installed on the remote end, or (with slight modification) if you happen to already have a tarball of the source directory. Feb 18, 2014 at 20:53
  • 1
    It needs a "p" (tar xpf) to preserve permissions at the destination.
    – Paul Lynch
    Sep 8, 2016 at 15:15
  • A note, this command must be run from the source computer, pushing the files into the target computer. For my case (many small, uncompressed files), it worked better and faster the scp or rsync. Jun 21, 2020 at 11:54
  • A follow-up question: on the source side, does the tar command store the tarball data locally first, before transferring to the remote? [This would be a problem when the source disk is almost full - say 98% full] Jan 7, 2021 at 16:46
  • What failed for rsync?
    – rogerdpack
    Nov 3, 2022 at 17:44
20

Using tar over ssh as both sender and receiver does the trick as well:

cd $DEST_DIR
ssh user@remote-host 'cd $REMOTE_SRC_DIR; tar cf - ./' | tar xvf -
2
  • I added on the source side --one-file-system and on the destination side p to preserve permissions.
    – ceving
    Jun 3, 2014 at 9:00
  • man page for tar says; --one-file-system = do not cross mount points Dec 15, 2016 at 0:17
2

One solution is to use a shell pipe. I have a situation where I got some *.gz files and symbolic links generated by some software to link to the same *.gz files with a slightly shorter name. If I simply use scp, then the symbolic links will be copied as regular files and resulting in duplicates. I know rsync can ignore symbolic links, but my gz files are not compressed with syncable options, and sync is very slow in copying these gz files. So I simply use the following script to copy over the files:

find . -type f -exec scp {} target_host:/directory/name/data \;

The -f option will only find regular files and ignore symbolic links. You need to give this command on the source host. Hope this may help some user in my situation. Let me know if I missed anything.

0

A one liner solution which can be executed at client to copy folder from server using tar + ssh command.

ssh user@<Server IP/link> 'mkdir -p <Remote destination directory;cd <Remote destination directory>; tar cf - ./' | tar xf - C <Source destination directory>

Note: mkdir is must, if the remote destination directory is not present then the command will simply compress the entire home of the remote server and extract it to client.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.