10

there are some videos that can't download.

I got download url by using this

http://www.youtube.com/get_video_info?video_id=itGNQbJwRSk

Is there anyone why below url is not available to download?

total url:

http://r1---sn-a5m7lnek.googlevideo.com/videoplayback?expire=1391267516&ms=au&source=youtube&sver=3&upn=OzZzFV_-2o4&id=8ad18d41b2704529&itag=22&mt=1391243224&ipbits=0&ratebypass=yes&fexp=935622%2C914924%2C926515%2C916623%2C936910%2C936913&sparams=gcr%2Cid%2Cip%2Cipbits%2Citag%2Cratebypass%2Csource%2Cupn%2Cexpire&mv=m&ip=183.101.166.55&key=yt5&gcr=kr&signature=F303D0C863C27A6A46124A09E40F308BB67181E013.3FA3E17460DDF6ECA004D9E48B1356849534EBFFFF

separated url with each params:

http://r1---sn-a5m7lnek.googlevideo.com/videoplayback?

expire=1391267516&

ms=au&

source=youtube&

sver=3&

upn=OzZzFV_-2o4&

id=8ad18d41b2704529&

itag=22&

mt=1391243224&

ipbits=0&

ratebypass=yes&

fexp=935622%2C914924%2C926515%2C916623%2C936910%2C936913&

sparams=gcr%2Cid%2Cip%2Cipbits%2Citag%2Cratebypass%2Csource%2Cupn%2Cexpire&

mv=m&

ip=183.101.166.55&

key=yt5&

gcr=kr&

signature=F303D0C863C27A6A46124A09E40F308BB67181E013.3FA3E17460DDF6ECA004D9E48B1356849534EBFFFF

and I used below codes in open stream

    u = new URL(url);
    HttpURLConnection huc = (HttpURLConnection)u.openConnection();//to know the size of video
    huc.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.102 Safari/537.36");
    huc.setRequestProperty("Accept","*/*");
    huc.setRequestMethod("GET");
    huc.setDoOutput(false);
    int size = huc.getContentLength();      
    int status = huc.getResponseCode();
    Log.e("download", status+ huc.getResponseMessage());
    is = new BufferedInputStream(huc.getInputStream());

but it returns status = 403, size = -1. I couldn't find anything to help yet.

Is there anyone have a idea?

2

5 Answers 5

13
+50

This video is using cipher signature. As you can see in the info returned by http://www.youtube.com/get_video_info?video_id=itGNQbJwRSk, it has

'use_cipher_signature=True'

To download this type of video, first you have to decode its signature. Decoding algorithm is present in the player file (HTML5player or Flash player).

After decoding you have to use the same IP address to download the video from where you have fetched all the information. (You can bypass same IP restriction using Proxy)

It's better to scrape the webpage of that particular video and search for ;ytplayer.config = {(.*?)}; . This contains all the required information.

If you don't want to scrape then you can use below method to get the JSON object:

https://www.youtube.com/watch?v=<Video-ID>&spf=prefetch

Please check this API for easy solution: CipherAPI.

Another cool API: YTstream API.

10
  • Also, I am able to download it by my method :)
    – Akhilesh
    Feb 10, 2014 at 16:15
  • Do I have to change ip address of parsed element to android's ip address?
    – Lion.k
    Feb 10, 2014 at 16:21
  • Yes. But did you decoded the cipher signature. If not then it will still throw 403 error. Also same IP address restriction can be bypassed by using a proxy.
    – Akhilesh
    Feb 10, 2014 at 16:31
  • 1
    yes, I decoded the signature using "&sig=" or "&s=" but I still get 403 error. Can you give me some reference site of bypassing Ip restriction?
    – Lion.k
    Feb 10, 2014 at 16:46
  • Tubget is first downloading the file on their server and then providing it to its users
    – Akhilesh
    Feb 10, 2014 at 17:07
4

In short: https://github.com/lure/YoutubeGrabber

Long story: Every youtube video page contains a js block which initialise html5 player. Of course, there is aa flash player, too, but JS one is more easily to understand.

Streams located in two params: url_encoded_fmt_stream_map for video+audio combined and adaptive_fmts for video or audio separate streams. Each link is made up from two let's call it parts:

  1. everything what is going before url=https: and
  2. everything from this point till the , (comma) which separates streams.

What you should know about initialising block is: some params may appear twice and you have to keep only one copy in your request. You can't predict which params may be dropped out so try to include everything you've found in a "raw" link. Special part of all of this is a signature that is required by backends. Beware, that signature may appear in a first or second part of link.

There is 3 types of signatures in a current time:

  • signature - this one is plain and need no decipher, requires no handling.
  • sig - ciphered
  • s - ciphered

Having this blocks, player should prepare each link and request backend for stream chosen. It's done by concatenating all existing params excluding duplicates and append prepared signature.

While you can carefully find all possible player version this likely is not the best solution. It seems better to download player from your application and extract the exact decoding function from it. This approach will work with ANY youtube video page.

Despite of your decision, you still need the function. Most of the time this function may be found with this RegEx

set\("signature",\s*(?:([^(]*).*)\);

Use any formatting tool to read it source. You'll find that the main function uses one addition so you should extract it too. After that it's quite easy to implement the function in java or (which I prefer better) to extract decipher function and keep it around.

Steps in short: 1. extract url_encoded_fmt_stream_map and adaptive_fmts 2. split by , 3. find decipher function in player and reimplement it in language of your choice or extract and eval.

for example, here is a function extracted from http://s.ytimg.com/yts/jsbin/html5player-new-en_US-vflhlPTtB/html5player-new.js

var fs = {
    Q2: function (a, b) {
        a.splice(0, b)
    }, cK: function (a) {
        a.reverse()
    }, yy: function (a, b) {
        var c = a[0];
        a[0] = a[b % a.length];
        a[b] = c
    }
};
function gs(a) {
    a = a.split("");
    fs.yy(a, 40);
    fs.Q2(a, 3);
    fs.yy(a, 53);
    fs.yy(a, 11);
    fs.Q2(a, 3);
    fs.cK(a, 8);
    fs.Q2(a, 3);
    fs.yy(a, 16);
    fs.cK(a, 75);
    return a.join("")
};
2
  • Thanks for your answer. I am able to extract url_encoded_fmt_stream_map and adaptive_fmts and also being able to find key value pairs within them. I can see some streams have key 'signature' and some have 's'. I need help to identify where can I find the decrypt routing. I have got base.js file and it has so many functions. So I am not sure if gs() and fs() is the final function list - or do we have to find some pattern from it. Also, what do I apply gs() on? is this like signature = gs(s.value) ? Mar 14, 2016 at 7:02
  • > where can I find the decrypt routing. In the youtube player itself. I've specified, that signatures are decoded in a player. gs and fs are obfuscated names. You can try to call signature = gs(s.value) it should work for some videos. If not, your best shot is to extract the appropriate function from javascript code and run it in JVM. Please, examine: github.com/lure/YoutubeGrabber/blob/master/src/main/scala/ru/…, the def registerPlayer function.
    – Observer
    Mar 15, 2016 at 9:08
1

I also Search on Same kind of stuff and find few useful links , on some videos they are not so good but hope after debug or see the code you may easily understand how they workd

What us Understand :

1-These code Download watch Url from link which is basicaly metadata of video 2-Parse that file according to format( video URL , Quality ) 3-Decrypt Urls

http://www.codeproject.com/Tips/323771/YouTube-Downloader-Using-Csharp-NET https://github.com/flagbug/YoutubeExtractor

1
0

I write a python script to download youtube video: https://github.com/wayne931121/youtube_downloader

There is one thing to attention. When you get the decrypt function in base.js, that's not mean you get the currect decrypt function, because in different youtube video may have different base.js url. Yesterday I think I successfully get the decrypt function in base.js, but it keep 403 forbidden and 403 forbidden and 403 forbidden... . I don't know why until today I find the decrypt function in base.js is different than yesterday, the yestrday I got var function Rwa(){...}, but today I got var function Qwa(a){a.split("");...} in base.js . Lucky, the real decrypt function is not much different than yesterday, it only change name, rearrange the real decrypt function in Rwa(a){a.split("")...}, and replace the arguments that the real decrypt function will use (int). And when I use the currect decrypt function, it successfully get the video for me. And I finnally know why youtube.dl use regex to catch decrypt function in base.js instead of direct let javascript decrypt function transform to python function.

It seems youtube change base.js url once a day.

When you want to find the decrypt function in base.js, the key words are: split(""), encodeURIComponent, decodeURIComponent, signature, sig.

You can also see the details in:

https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/youtube.py#L1403

Here are some base.js url I got.

Format:

Windows:

https://www.youtube.com/s/player/${Js_Id}/player_ias.vflset/${Country}/base.js

Android:

https://www.youtube.com/s/player/${Js_Id}/player-plasma-ias-tablet-${Country}/base.js

Some url I catched: (2022/10/${day})

https://www.youtube.com/s/player/4bbf8bdb/player_ias.vflset/zh_TW/base.js
https://www.youtube.com/s/player/24c6f8bd/player_ias.vflset/zh_TW/base.js https://www.youtube.com/s/player/64588dad/player_ias.vflset/zh_TW/base.js https://www.youtube.com/s/player/4bbf8bdb/player-plasma-ias-tablet-zh_TW.vflset/base.js https://www.youtube.com/s/player/24c6f8bd/player-plasma-ias-tablet-zh_TW.vflset/base.js https://www.youtube.com/s/player/64588dad/player-plasma-ias-tablet-zh_TW.vflset/base.js

Something I finded in base.js:

//From youtube base.js

//Yesterday I got
Rwa=function(a){a=a.split("");RB.ss(a,4);RB.jM(a,6);RB.jM(a,45);RB.e0(a,3);return a.join("")};
var RB={ss:function(a){a.reverse()},
jM:function(a,b){var c=a[0];a[0]=a[b%a.length];a[b%a.length]=c},
e0:function(a,b){a.splice(0,b)}};

//Today I got
Qwa=function(a){a=a.split("");PB.Co(a,14);PB.Co(a,14);PB.Zo(a,56);PB.GZ(a,2);return a.join("")};
var PB={Co:function(a,b){var c=a[0];a[0]=a[b%a.length];a[b%a.length]=c},
Zo:function(a){a.reverse()},
GZ:function(a,b){a.splice(0,b)}};

//Common
QE = function(a, b, c) {  //### This Function will decrypt signatureChiper ###//
    b = void 0 === b ? "" : b;
    c = void 0 === c ? "" : c;
    a = new g.SB(a,!0);
    a.set("alr", "yes");
    c && (c = Rwa(decodeURIComponent(c)), //### This Line will decrypt signatureChiper ###//
    a.set(b, encodeURIComponent(c)));
    return a
}

QE = function(a, b, c) {  //### This Function will decrypt signatureChiper ###//
    b = void 0 === b ? "" : b;
    c = void 0 === c ? "" : c;
    a = new g.SB(a,!0);
    a.set("alr", "yes");
    c && (c = Qwa(decodeURIComponent(c)), //### This Line will decrypt signatureChiper ###//
    a.set(b, encodeURIComponent(c)));
    return a
}

RE=function(a,b,c){b=void 0===b?"":b;c=void 0===c?"":c;a=new g.SB(a,!0);a.set("alr","yes");c&&(c=Rwa(decodeURIComponent(c)),a.set(b,encodeURIComponent(c)));return a};
-1

You can't download video from youtube. It's illegal. You can only use the playback services.

Section 4c You agree not to access Content through any technology or means other than the video playback pages of the Service itself, the Embeddable Player, or other explicitly authorized means YouTube may designate.

4
  • This site "tubget.com" can download every youtube video. What I want to know is How they do this.
    – Lion.k
    Feb 6, 2014 at 6:14
  • I dont think you would get anything illegal from this site Feb 6, 2014 at 6:43
  • 3
    This is not an answer to the question that was asked. This should be a comment.
    – Synetech
    May 17, 2015 at 2:27
  • 8
    Being against a website's ToS does not make it illegal.
    – szr
    Aug 20, 2016 at 17:39

Not the answer you're looking for? Browse other questions tagged or ask your own question.