65

I want to mirror a folder via FTP, like this:

wget --mirror --user=x --password=x ftp://ftp.site.com/folder/subfolder/evendeeper

But I do not want to create a directory structure like this:

ftp.site.com -> folder -> subfolder -> evendeeper

I just want:

evendeeper

And anything below it to be the resulting structure. It would also be acceptable for the contents of evendeeper to wind up in the current directory as long as subdirectories are created for subdirectories of evendeeper on the server.

I am aware of the -np option, according to the documentation that just keeps it from following links to parent pages (a non-issue for the binary files I'm mirroring via FTP). I am also aware of the -nd option, but this prevents creating any directory structure at all, even for subdirectories of evendeeper.

I would consider alternatives as long as they are command-line-based, readily available as Ubuntu packages and easily automated like wget.

1
  • With lftp: lftp -c mirror <url>
    – HappyFace
    Aug 29, 2021 at 16:07

4 Answers 4

89

For a path like: ftp.site.com/a/b/c/d

-nH would download all files to the directory a/b/c/d in the current directory, and -nH --cut-dirs=3 would download all files to the directory d in the current directory.

5
  • 4
    Thanks, --cut-dirs was the silver bullet I was looking for. Feb 20, 2011 at 13:33
  • 5
    I would change to --cut-dirs=4 if you need all files from directory d
    – Mike
    Oct 8, 2013 at 14:40
  • 5
    -nH would only ignore the host directory for me. You can set --cut-dirs=100 for preventing almost all the pre directorys -- tick
    – Rex
    Feb 19, 2016 at 9:10
  • 5
    It's fairly obvious with an URL like ftp.site.com/a/b/c/d/ one only wants the content in d/ and its subfolders. Is there an auto-cut option or something similar for wget, that autodetects the cut depth?
    – con-f-use
    Mar 14, 2016 at 12:48
  • 2
    @con-f-use I was hoping the --cut-dirs=100 trick suggested by @Rex (and in Tony TCG's answer below) would achieve this, but alas, it works like -nd, i.e. it flattens the entire hierarchy, including subdirectories below the one you're downloading. Looks like there's no automatic solution using wget alone...
    – waldyrious
    Oct 4, 2017 at 14:13
11

I had a similar requirement and the following combination seems to be the perfect choice:

In the below example, all the files in http://url/dir1/dir2 (alone) are downloaded to local directory /dest/dir

wget  -nd -np -P /dest/dir --recursive http://url/dir1/dir2

Thanks @ffledgling for the hint on "-nd"

For the above example:

wget -nd -np --mirror --user=x --password=x ftp://ftp.site.com/folder/subfolder/evendeeper

Snippets from manual:

   -nd
   --no-directories
       Do not create a hierarchy of directories when retrieving recursively.  With this option turned on, all files will get saved to the current directory, without clobbering (if a name shows up more than once, the
       filenames will get extensions .n).


   -np
   --no-parent
       Do not ever ascend to the parent directory when retrieving recursively.  This is a useful option, since it guarantees that only the files below a certain hierarchy will be downloaded.
10

-np (no parent) option will probably do what you want, tied in with -L 1 (I think, don't have a wget install before me), which limits the recursion to one level.

EDIT. ok. gah... maybe I should wait until I've had coffee.. There is a --cut or similar option, which allows you to "cut" a specified number of directories from the output path, so for /a/b/c/d, a cut of 2 would force wget to create c/d on your local machine

1
  • 6
    You mean: --cut-dirs=2 — Also, -L is --relative, so you probably meant -l or --level=1 Feb 18, 2011 at 15:51
9

Instead of using:

-nH --cut-dirs=1

use:

-nH --cut-dirs=100

This will cut more directories and no folders will be created.

Note: 100 = the number of folders to skip creating. You can change 100 to any number.

1
  • 10
    Why not just use -nd instead then?
    – ffledgling
    Mar 31, 2015 at 1:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.