95

When downloading a file using curl, how would I follow a link location and use that for the output filename (without knowing the remote filename in advance)?

For example, if one clicks on the link below, you would download a filenamed "pythoncomplete.vim." However using curl's -O and -L options, the filename is simply the original remote-name, a clumsy "download_script.php?src_id=10872."

curl -O -L http://www.vim.org/scripts/download_script.php?src_id=10872

In order to download the file with the correct filename you would have to know the name of the file in advance:

curl -o pythoncomplete.vim -L http://www.vim.org/scripts/download_script.php?src_id=10872

It would be excellent if you could download the file without knowing the name in advance, and if not, is there another way to quickly pull down a redirected file via command line?

10 Answers 10

176

The remote side sends the filename using the Content-Disposition header.

curl 7.21.2 or newer does this automatically if you specify --remote-header-name / -J.

curl -O -J -L $url

The expanded version of the arguments would be:

curl --remote-name --remote-header-name --location $url
12
  • 17
    This is not working for me, the saved file name is still in download_script.php?src_id=10872 format. I am running curl 7.30.0 on my macbook pro
    – ilight
    Nov 1, 2013 at 6:49
  • 1
    Work fine with curl 7.38.0
    – ismail
    Oct 8, 2014 at 17:20
  • 3
    Not working with curl on OSX: curl 7.42.1 (x86_64-apple-darwin14.4.0) libcurl/7.42.1 SecureTransport zlib/1.2.5 libssh2/1.5.0 May 31, 2015 at 21:40
  • 4
    As @Diskutant "comments in an-answer" below. This fails in curl in the somewhat common case of Content-Disposition: attachment; filename = foo.txt. The spaces around the = cause it (see: gist.github.com/jnewman/23c993ef50bcd69f9086fcd4e2594928)
    – fncomp
    Apr 7, 2016 at 20:45
  • 2
    In case someone else comes across this if you are using localhost you have to make it http://localhost or the -J is ignored
    – mvndaai
    Nov 17, 2016 at 19:10
35

If you have a recent version of curl (7.21.2 or later), see @jmanning2k's answer.

I you have an older version of curl (like 7.19.7 which came with Snow Leopard), do two requests: a HEAD to get the file name from response header, then a GET:

url="http://www.vim.org/scripts/download_script.php?src_id=10872"
filename=$(curl -sI  $url | grep -o -E 'filename=.*$' | sed -e 's/filename=//')
curl -o $filename -L $url
5
  • 5
    Thanks! It came back with a question mark at the end of the filename, so I added a tr -d '\r\n' between your grep and sed commands, and it worked marvelously.
    – Nick S.
    Jul 30, 2011 at 20:03
  • In cygwin, I get a file name called -L :(. I used curl --remote-header-name --remote-name URL which seemed to work.
    – Sun
    Sep 27, 2014 at 7:33
  • 2
    WIth curl 7.26.0 I could get file name by doing ´curl -sI $url | grep -oP '(?<=Location: )[^\s]+' | xargs basename´.
    – aesede
    Sep 15, 2015 at 1:47
  • 2
    The first curl would probably also need the -L option.
    – TNT
    Jun 12, 2017 at 15:37
  • 2
    I had to use filename=$(curl -sI $url | grep -o -E 'location:.*$' | sed -e 's/location:=//' | xargs basename)
    – hoijui
    May 28, 2019 at 8:41
20

If you can use wget instead of curl:

wget --content-disposition $url
2
  • This works for a url such as: plex.tv/downloads/latest/… Sep 7, 2018 at 22:14
  • 1
    You are a f-ing legend!! I was monkey around with this A LOT and this worked out of the gates. Thanks. ProTip: If you have a lot of URLs, you can create a file like urls.txt, fill it with URLs (one per line, no surrounding quotes) and then use wget --content-disposition -i urls.txt to download them all in sequence. Boom! Jan 10, 2022 at 20:45
12

I wanted to comment to jmanning2k's answer but as a new user I can't, so I tried to edit his post which is allowed but the edit was rejected saying it was supposed to be a comment. sigh

Anyway, see this as a comment to his answer thanks.

This seems to only work if the header looks like filename=pythoncomplete.vim as in the example, but some sites send a header that looks like filename*=UTF-8' 'filename.zip' that one isn't recognized by curl 7.28.0

6

I wanted a solution that worked on both older and newer Macs, and the legacy code David provided for Snow Leopard did not behave well under Mavericks. Here's a function I created based on David's code:

function getUriFilename() {
    header="$(curl -sI "$1" | tr -d '\r')"

    filename="$(echo "$header" | grep -o -E 'filename=.*$')"
    if [[ -n "$filename" ]]; then
        echo "${filename#filename=}"
        return
    fi

    filename="$(echo "$header" | grep -o -E 'Location:.*$')"
    if [[ -n "$filename" ]]; then
        basename "${filename#Location\:}"
        return
    fi

    return 1
}

With this defined, you can run:

url="http://www.vim.org/scripts/download_script.php?src_id=10872"
filename="$(getUriFilename $url)"
curl -L $url -o "$filename"
2

Please note that certain malconfigured webservers will serve the name using "Filename" as key, where RFC2183 specifies it should be "filename". curl only handles the latter case.

2

I had the same Problem like John Cooper. I got no filename but a Location File name back. His answer also worked but are 2 commands. This oneliner worked for me....

url="https://download.mozilla.org/?product=firefox-latest-ssl&os=linux64&lang=de";url=$(curl -L --head -w '%{url_effective}' $url 2>/dev/null | tail -n1) ; curl -O $url

Stolen and added some stuff from https://unix.stackexchange.com/questions/126252/resolve-filename-from-a-remote-url-without-downloading-a-file

1
  • this question gave me a lot of headache, and I have found no solution rather than: 1) using wget -O or ... using this answer!!! My opinion, THIS IS THE SOLUTION! I will put therefore a function below, to avoid further headaches... May 19, 2021 at 21:30
1

An example using the answer above for Apache Archiva artifact repository to pull latest version. The curl returns the Location line and the filename is at the end of the line. Need to remove the CR at end of file name.

url="http://archiva:8080/restServices/archivaServices/searchService/artifact?g=com.imgur.backup&a=snapshot-s3-util&v=LATEST"
filename=$(curl --silent -sI -u user:password $url | grep Location | awk -F\/ '{print $NF}' | sed 's/\r$//')
curl --silent -o $filename -L -u user:password $url
1

instead of applying grep and other Unix-Fu operations, curl ships with a builtin "Write Out" option variable[1] specifically for such a case, e.g.

$ curl -OJsL "http://www.vim.org/scripts/download_script.php?src_id=10872" -w "%{filename_effective}"
pythoncomplete.vim

[1] https://everything.curl.dev/usingcurl/verbose/writeout#available-write-out-variables

1
  • If the goal is to just save the file with a correct filename, using -w and this write-out variable to output file name to stdout isn't needed. The key option is -O (--remote-name) which tells that saved file should have the same name as remote. See the accepted answer.
    – YurkoFlisk
    Aug 6, 2022 at 1:16
0

Using the solution proposed above, I wrote this helper function curl2file.

[UPDATED]

function curl2file() {
    url=$1
    url=$(curl -o /dev/null -L --head -w '%{url_effective}' $url 2>/dev/null | tail -n1) ; curl -O $url
}

Usage:

curl2file https://cloud.tsinghua.edu.cn/f/4666d28af98a4e63afb5/?dl=1
1
  • 1
    You should use -o /dev/null when getting the redirect URL so that you don't need to process the output (-o affects the --head output but not the -w output). Also, you should not redirect stderr because you then hide any errors that occur.
    – cjs
    Sep 8, 2022 at 3:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.