192

I need to get the final URL after a page redirect preferably with curl or wget.

For example http://google.com may redirect to http://www.google.com.

The contents are easy to get(ex. curl --max-redirs 10 http://google.com -L), but I'm only interested in the final url (in the former case http://www.google.com).

Is there any way of doing this by using only Linux built-in tools? (command line only)

11 Answers 11

342

curl's -w option and the sub variable url_effective is what you are looking for.

Something like

curl -Ls -o /dev/null -w %{url_effective} https://example.com

More info

-L         Follow redirects
-s         Silent mode. Don't output anything
-o FILE    Write output to <file> instead of stdout
-w FORMAT  What to output after completion

More

You might want to add -I (that is an uppercase i) as well, which will make the command not download any "body", but it then also uses the HEAD method, which is not what the question included and risk changing what the server does. Sometimes servers don't respond well to HEAD even when they respond fine to GET.

12
  • 5
    you should be able to use "-o /dev/null" if you don't want the file Commented Jun 20, 2010 at 17:38
  • 2
    That's more of a shell feature than curl
    – user151841
    Commented May 31, 2012 at 18:39
  • 3
    @DanielStenberg you need -I otherwise it will actually download the file.
    – Zombo
    Commented Jun 15, 2014 at 2:05
  • doesn't work. curl -Ls -o /dev/null -w %{url_effective} https://goo dot gl/un5E outputs https://goo dot gl/un5E (replace dot with .)
    – Toolkit
    Commented May 4, 2017 at 14:22
  • 2
    Some websites also need a spoofed user agent with curl -A ... to redirect to the expected location.
    – Ivan Kozik
    Commented Oct 8, 2018 at 13:59
42

Thanks, that helped me. I made some improvements and wrapped that in a helper script "finalurl":

#!/bin/bash
curl "$1" -s -L -I -o /dev/null -w '%{url_effective}'
  • -o output to /dev/null
  • -I don't actually download, just discover the final URL
  • -s silent mode, no progressbars

This made it possible to call the command from other scripts like this:

echo `finalurl http://someurl/`
1
  • 6
    Thanks for those ideas. I rewrote it for terminal usage in my .bashrc file as a function, and there's no need for the terse options in that file, so I used the long names to self-document this: finalurl() { curl --silent --location --head --output /dev/null --write-out '%{url_effective}' -- "$@"; }
    – user964843
    Commented Feb 10, 2017 at 18:14
11

as another option:

$ curl -i http://google.com
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Sat, 19 Jun 2010 04:15:10 GMT
Expires: Mon, 19 Jul 2010 04:15:10 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219
X-XSS-Protection: 1; mode=block

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

But it doesn't go past the first one.

6

Thank you. I ended up implementing your suggestions: curl -i + grep

curl -i http://google.com -L | egrep -A 10 '301 Moved Permanently|302 Found' | grep 'Location' | awk -F': ' '{print $2}' | tail -1

Returns blank if the website doesn't redirect, but that's good enough for me as it works on consecutive redirections.

Could be buggy, but at a glance it works ok.

6

You can do this with wget usually. wget --content-disposition "url" additionally if you add -O /dev/null you will not be actually saving the file.

wget -O /dev/null --content-disposition example.com

2
  • Replace from -O /dev/null to only -O-. Better: wget -O- --content-disposition example.com Commented Jun 22, 2019 at 0:11
  • 1
    wget -O /dev/null --content-disposition example.com and wget -O- /dev/null --content-disposition example.com produce a lot more output than the redirected URL. curl $1 -s -L -I -o /dev/null -w '%{url_effective}' works fine for me.
    – Eric Klien
    Commented Jun 22, 2019 at 7:08
5

The parameters -L (--location) and -I (--head) still doing unnecessary HEAD-request to the location-url.

If you are sure that you will have no more than one redirect, it is better to disable follow location and use a curl-variable %{redirect_url}.

This code do only one HEAD-request to the specified URL and takes redirect_url from location-header:

curl --head --silent --write-out "%{redirect_url}\n" --output /dev/null "https://""goo.gl/QeJeQ4"

Speed test

all_videos_link.txt - 50 links of goo.gl+bit.ly which redirect to youtube

1. With follow location

time while read -r line; do
    curl -kIsL -w "%{url_effective}\n" -o /dev/null  $line
done < all_videos_link.txt

Results:

real    1m40.832s
user    0m9.266s
sys     0m15.375s

2. Without follow location

time while read -r line; do
    curl -kIs -w "%{redirect_url}\n" -o /dev/null  $line
done < all_videos_link.txt

Results:

real    0m51.037s
user    0m5.297s
sys     0m8.094s
2
  • 1
    Seems pretty uncommon that you'd know in advance that there would only be one redirect ...
    – SamB
    Commented Oct 7, 2019 at 16:54
  • Thanks, actually your method was the only one that was able to give me the correct url from: breachbase.pwnedbook.com/leaks/facebook of all examples in this thread, that page redirects to a youtube url and is a "test" for see how many that is willing to click on the url for download a breach without checking the end of it, however thanks to you i didn't become a victim of this test as 1 trillion others have become, according to the number of viewers on the youtube vid.. Respect! +1 of course. // wuseman
    – wuseman
    Commented Sep 2, 2022 at 23:25
4

curl can only follow http redirects. To also follow meta refresh directives and javascript redirects, you need a full-blown browser like headless chrome:

#!/bin/bash
real_url () {
    printf 'location.href\nquit\n' | \
    chromium-browser --headless --disable-gpu --disable-software-rasterizer \
    --disable-dev-shm-usage --no-sandbox --repl "$@" 2> /dev/null \
    | tr -d '>>> ' | jq -r '.result.value'
}

If you don't have chrome installed, you can use it from a docker container:

#!/bin/bash
real_url () {
    printf 'location.href\nquit\n' | \
    docker run -i --rm --user "$(id -u "$USER")" --volume "$(pwd)":/usr/src/app \
    zenika/alpine-chrome --no-sandbox --repl "$@" 2> /dev/null \
    | tr -d '>>> ' | jq -r '.result.value'
}

Like so:

$ real_url http://dx.doi.org/10.1016/j.pgeola.2020.06.005 
https://www.sciencedirect.com/science/article/abs/pii/S0016787820300638?via%3Dihub
1
  • On Chrome (not Chromium) this is basically useless.
    – Ken Sharp
    Commented Sep 9 at 7:55
3

This would work:

 curl -I somesite.com | perl -n -e '/^Location: (.*)$/ && print "$1\n"'
1

I'm not sure how to do it with curl, but libwww-perl installs the GET alias.

$ GET -S -d -e http://google.com
GET http://google.com --> 301 Moved Permanently
GET http://www.google.com/ --> 302 Found
GET http://www.google.ca/ --> 200 OK
Cache-Control: private, max-age=0
Connection: close
Date: Sat, 19 Jun 2010 04:11:01 GMT
Server: gws
Content-Type: text/html; charset=ISO-8859-1
Expires: -1
Client-Date: Sat, 19 Jun 2010 04:11:01 GMT
Client-Peer: 74.125.155.105:80
Client-Response-Num: 1
Set-Cookie: PREF=ID=a1925ca9f8af11b9:TM=1276920661:LM=1276920661:S=ULFrHqOiFDDzDVFB; expires=Mon, 18-Jun-2012 04:11:01 GMT; path=/; domain=.google.ca
Title: Google
X-XSS-Protection: 1; mode=block
1

Can you try with it?

#!/bin/bash 
LOCATION=`curl -I 'http://your-domain.com/url/redirect?r=something&a=values-VALUES_FILES&e=zip' | perl -n -e '/^Location: (.*)$/ && print "$1\n"'` 
echo "$LOCATION"

Note: when you execute the command curl -I http://your-domain.com have to use single quotes in the command like curl -I 'http://your-domain.com'

-5

You could use grep. doesn't wget tell you where it's redirecting too? Just grep that out.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.