I need to copy all the .class files from server to local with all dir reserved. e.g. server:/usr/some/unknown/number/of/sub/folders/me.class
will be /usr/project/backup/some/unknown/number/of/sub/folders/me.class
the problem is, there are many other useless files such as .svn-base files that i don't want. how can i filter them so I only scp
.class files?
-
I like the rsync option mentioned. You didn't mention if this is a one-off operation, or if you'll be automating this repeatedly. For a one-off operation, the judicious use of find, grep -v, xargs and temporary files should make short work of this.– user47559Aug 4, 2009 at 16:40
9 Answers
I'd probably recommend using something like rsync
for this due to its include
and exclude
flags, e.g:-
rsync -rav -e ssh --include '*/' --include='*.class' --exclude='*' \
server:/usr/some/unknown/number/of/sub/folders/ \
/usr/project/backup/some/unknown/number/of/sub/folders/
Some other useful flags:
-r
for recursive-a
for archive (mostly all files)-v
for verbose output-e
to specify ssh instead of the default (which should be ssh, actually)
-
4Anyway to make this ignore subfolders that don't have *class files in them? (i.e. I don't want a bunch of empty dirs) Oct 25, 2013 at 15:09
-
-
2Can you explain --include, not --include= In the MAN pages, I could find explanation on --include= but not --include– bazzFeb 10, 2014 at 4:15
-
2The option
-a
already includes-r
according to the rsync man pages. Oct 26, 2017 at 10:57 -
4
To exclude dotfiles in base directory:
scp -r [!.]* server:/path/to/something
[!.]*
is a shell glob that expands to all files in working directory not starting with a dot.
-
10This is probably the best answer; using globs to filter files is the way to go. Dec 3, 2015 at 0:33
-
1If your option are restricted to scp like I had , this definitely helped. Dec 12, 2015 at 18:23
-
3This indicates to exclude files like being asked, but how can this be achieved for a whole directory?– PilleMay 30, 2017 at 16:14
-
1
There is no feature in scp to filter files. For "advanced" stuff like this, I recommend using rsync:
rsync -av --exclude '*.svn' user@server:/my/dir .
(this line copy rsync from distant folder to current one)
Recent versions of rsync tunnel over an ssh connection automatically by default.
Since you can scp
you should be ok to ssh
,
either script the following or login and execute...
# After reaching the server of interest
cd /usr/some/unknown/number/of/sub/folders
tar cfj pack.tar.bz2 $(find . -type f -name *.class)
return back (logout) to local server and scp
,
# from the local machine
cd /usr/project/backup/some/unknown/number/of/sub/folders
scp you@server:/usr/some/unknown/number/of/sub/folders/pack.tar.bz2 .
tar xfj pack.tar.bz2
If you find the $(find ...)
is too long for your tar change to,
find . -type f -name *.class | xargs tar cfj pack.tar.bz2
Finally, since you are keeping it in /usr/project/backup/
,
why bother extraction? Just keep the tar.bz2
, with maybe a date+time stamp.
-
This helped me. I'm in Windows and I'm fighting trying get
rsync
andssh
installed on the OS. Instead, this was a clever workaround. Thanks!– rayryengFeb 5, 2018 at 13:24
Below command for files.
scp `find . -maxdepth 1 -name "*.log" \! -name "hs_err_pid2801.log" -type f` root@IP:/tmp/test/
- IP will be destination server IP address.
- -name "*.log" for include files.
- \! -name "hs_err_pid2801.log" for exclude files.
- . is current working dir.
- -type f for file type.
Below command for directory.
scp -r `find . -maxdepth 1 -name "lo*" \! -name "localhost" -type d` root@IP:/tmp/test/
you can customize above command as per your requirement.
-
This will copy files from local to remote, the question was about remote to local Apr 4, 2021 at 12:36
With ssh key based authentication enabled, the following script would work.
for x in `ssh user@remotehost 'find /usr/some -type f -name *.class'`; do y=$(echo $x|sed 's/.[^/]*$//'|sed "s/^\/usr//"); mkdir -p /usr/project/backup$y; scp $(echo 'user@remotehost:'$x) /usr/project/backup$y/; done
If you indeed wanna use scp, there's a indirect way.Say we want to copy all .jpg file under local folder '/src' to folder '/dst' in remote server 10.1.1.2:
#make a clean temp folder
mkdir /tmp/ttt
#copy all .jpg file and retain folder structure as-is
find /src -type f -name *.jpg -exec cp --parents \{\} /tmp/ttt \;
#copy to remote target folder as-is and retain original time attributes
scp -rp /tmp/ttt/* 10.1.1.2:/dst
#if copy ok, remove temp folder
rm -rf /tmp/ttt
scp -i /home/<user>/.ssh/id_rsa -o "StrictHostKeyChecking=no" -rp /source/directory/path/[!.]* <target_user>@<target_system:/destination/directory/path
-
2
Copy your source folder to
somedir
:cp -r
srcdir
somedir
Remove all unneeded files:
find somedir -name '.svn' -exec rm -rf {} \+
launch scp from
somedir
-
why all the downvotes on this answer? This sounds like a via solution to the question.– bcarrollAug 28, 2014 at 13:30
-
8Because this would copy a bunch of unnecessary files then delete them, wasting potentially a lot of time.– OdedAug 14, 2015 at 6:33
-
For a program that goes through several compilation steps, this actually makes a lot of sense. It could go right before the step that packs the compiled code into a
tar
orzip
-file. Instead, there's no compression (scp does the compression) and the 'packaged' code gets copied over.– samvvMay 24, 2016 at 18:08 -
-
if files are being ignored because of space constraints, this wastes space as well as time– jakeMay 11, 2020 at 13:33