34

I'm trying to get a full list of watched videos for a given user in my YouTube API application. I want to add up total duration of all videos.

When I get the list of videos from history playlist, the API caps it at 50 items. There's pagination but total amount of items is 50 (not just per page); I can't access more data with the API it appears.

Is there any way I can get this playlist without the data cap? I'm hoping for another method (of using the API) or a way to do it without the API. I know YouTube stores this data because I can view my entire history (far more that 50 videos).

I'm using this code:

var requestOptions = {
    playlistId: playlistId,
    part: 'snippet',
    maxResults: 50
};
gapi.client.youtube.playlistItems.list(requestOptions);

where playlistId is the id of the history playlist I got from a gapi.client.youtube.channels.list request.

Edit (2017): I want to clarify that it was always my intention to download my own history, just out of interest to see how much time I have spent watching videos. I still have not been able to do this.

8
  • The documentation specifically says the acceptable values for maxResults are 0 to 50. So you'll have to tap into the paging
    – Ian
    Commented May 15, 2013 at 4:47
  • @Ian, there is no paging, there are only 50 results total. Like the response.result.pageInfo.totalResults which is the total number of videos across all pages is 50 as well. Furthermore it doesn't return a nextPageToken or a prevPageToken. Also this comment by an employee of Google confirms that it is capped.
    – Fsmv
    Commented May 15, 2013 at 5:37
  • 2
    If it's capped, they don't want you to do that. Finding workarounds, if at all possible, is likely a violation of their terms.
    – Ingo Bürk
    Commented May 15, 2013 at 6:17
  • In particular, the Youtube API Terms of Service II.2, which states that you must not interefere with the proper workings of the API. And if there is an intentional limit of 50 results, this is the proper workings. And you'll find this in any social network API, i.e. facebook.
    – Ingo Bürk
    Commented May 15, 2013 at 17:35
  • 2
    It seems that it's a BUG! Only the last "week or so" is displayed. Please support this Issue commenting: code.google.com/p/gdata-issues/issues/detail?id=4642
    – user1447667
    Commented Jul 16, 2015 at 0:14

6 Answers 6

17
+50

The API currently only retrieves the last two weeks of Watch History. For more information refer to the Bug Issue reported: https://code.google.com/p/gdata-issues/issues/detail?id=4642

Note: There is a similar question on SO asked here: YouTube API v3 returns truncated watch history

17

I wrote a scraper(in Python 2.7(updated for 3.5) and Scrapy) for this task a while ago. Sans official API, it uses a logged in session cookie and html parsing. Dumps to SQLite by default. https://github.com/zvodd/Youtube-Watch-History-Scraper

How it's done: essentially it opens the url

https://www.youtube.com/feed/history'

with a valid(logged in) session cookie taken from Chrome. Scrapes all video entries for name, vid(url), channel/user, description, length. Then it finds the button at the bottom of the page with the attribute data-uix-load-more-href which contains the link to the next page, something like:

"/browse_ajax?action_continuation=1&continuation=98h32hfoasau0fu928hf2hf908h98hr%253D%253D&target_id=item-section-552363&direct_render=1"

... re-scrapes the video entries from there and dumps them all into an sqlite database; which you can search entries by any of the fields (name, length, user, description, etc).

So until they change their feed/history page, it's doable and done. I might even update it.

3
  • Explicitly against the YouTube terms of service. Commented Aug 20, 2020 at 14:04
  • 1
    You're not wrong, but I doubt they care if people are using a scraper to get their own watch history. Most of the policy dot points surrounding that term are specifically around malicious use. So it probably does violate the letter of the policy but not the spirit of the policy. Theoretically "access the Service using any automated means" would also be violated if you used a keyboard macro whilst accessing site.
    – Zv_oDD
    Commented Aug 21, 2020 at 6:00
  • If you are a freelancer working for a corporate client, you should think twice before breaching the terms of service of anyone, as your insurance company may cover you once, but pretty sure no-one will cover you twice without a very large premium. Commented Aug 24, 2020 at 13:22
3

While this isn't currently possible using just the YouTube API, there is an (albeit slightly involved) method to calculate your watch time):

  1. download a list of your watch history as a JSON file using Google Takeout.
  2. Unfortunately the JSON file doesn't include the video durations, so the next step is to extract the video IDs (the part after "watch?v=" in the "titleURL" object
  3. Now take your list of video IDs, and send a request to the youtube API that looks something like this:
 function execute() {
    return gapi.client.youtube.videos.list({
      "part": [
        "contentDetails"
      ],
      "id": [
        "VIDEO IDs"
      ],
      "fields": "items(contentDetails(duration))"
    })

(Code created using YouTube API Explorer)

Note: You may need to break the list of video IDs into smaller lists (I had to) or the API may reject the request. As [pointed out by stvar in the comments] the ID list maximum length is 50, so this is the maximum length your lists can be. (full disclosure: I was using Python to send the requests)

  1. Finally, just extract the duration values and add them up (though this might not be quite as easy as it sounds)

The best part of this is I don't believe this actually violates any ToS.

2
  • The maximum length for the comma-separated video IDs list is 50; this value is derived from the fact that the API (by design) won't return more then 50 items within the result sets provided to its caller.
    – stvar
    Commented Nov 9, 2020 at 10:00
  • Indeed, I concur with your assessment: the solution above is DTOS-compliant.
    – stvar
    Commented Nov 9, 2020 at 10:02
3

It seems like this is a known bug originally reported in 2013. The exact same behavior is explained on a Google Code thread: https://code.google.com/p/gdata-issues/issues/detail?id=4642

0

Brainstorming, never tried: Have you tried not using the API and instead parsing the https://www.youtube.com/feed/history URL?

Theoretically, the user browsing could be emulated, including the pagination. I am not aware of how hard though (probably very), since you need to deal with authentication and YouTube probably tries to verify that a human is browsing.

2
  • 2
    See this comment against the TOS is a no-go
    – Fsmv
    Commented Jul 28, 2015 at 21:59
  • 1
    You wouldn't be using the API, but a regular HTTP access, parsing the responses and emulating a user interaction ( something like curl ... GET ... https://www.youtube.com/feed/history ). This is probably a pain, but not an abuse of the API. Perhaps an abuse of the TOS of a regular user.
    – noderman
    Commented Jul 29, 2015 at 2:10
0

I was looking for some way to get the list of YouTube history. I just found out that Google has a tool for this. In Google Takeout you have a option taht you can get the entire list of watched videos. My list went back util 2011.

To get explanation short there are two videos explaining how to do this:

https://www.youtube.com/watch?v=zlzzO1e6dws https://www.youtube.com/watch?v=dto8jGMxHxY

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.