24

Is there a way to determine the intended frame rate of content playing in the html video element?

Does the video element even know the intended FPS or frame count, or does it simply "guess" (maybe 24fps) and play at the guessed speed?

Here are my unsuccessful attempts:

  • Look for a FPS or FrameCount property on the video element itself--Nope!

  • Look for cross-video-format header info about FPS or FrameCount--Nothing consistent!

  • Look for an event that is triggered upon frame changing--Nope!

My next attempt is more complicated: Sample the video by capturing frames to a canvas element and count frames by determining when the pixels change.

Does anyone have a simpler answer before I do the complicated attempt?

2
  • the mp4 container (MOOV atom) does contain the target framerate, so I would assume the useragent reads that and uses it to control target playback rate Feb 10, 2015 at 0:42
  • @Offbeatmammal, Thanks, that property on mp4 is useful even if the browser doesn't use it to control the playback rate. However, it's a partial solution because I couldn't find similar framerate properties on other encodings. :-/
    – markE
    Feb 10, 2015 at 18:33

3 Answers 3

27

Knowing the frame-rate of the video wouldn't be as useful as you might think.
Browsers uses of some tricks to make a match between the frame-rate of the movie and the refresh-rate of the screen, so if you look at currentTime property, you'll see that the actual frame duration ( == currentTime - previous currentTime) is not a constant, it varies from frame to frame.

On this sample video : http://jsfiddle.net/3qs46n4z/3/ the pattern is :
4-1-5-1 :
4 frames at 21.3 + 1 frame at 32 + 5 frames at 21.3 + 1 frame at 32.

So if you want to always display the latest frame on a canvas while avoiding overdraw, the solution might be to :
- On each rAF, look at the current time of the video :
• Same ? -> do nothing.
• Changed ? -> update frame.

And whatever you want to do, comparing two currentTime === two numbers might be faster than comparing two imageDatas ;-)

Edit : looking into the specifications to find evidence of my saying, i found a nuance with this Note :

 Which frame in a video stream corresponds to a particular playback position is defined by the video stream's format.

(Note of 4.8.6 at http://www.w3.org/TR/2011/WD-html5-20110113/video.html )

So strictly saying we can only say that (the current time is the same) implies (the frames are identical).
I can only bet that the reciprocal is true => different time means different frame.
In the example above, Chrome is trying to match the 24Hz of the movie on my 60Hz computer by trying to get 45 Hz ( = 60 / 2 + 60 / 4), the nearest from 48 = 2*24. For the 21 created frames i don't know if it interpolates or merely duplicates the frames. It surely changes depending on browser/device (Gpu especially). I bet any recent desktop or powerful smartphone does interpolate.

Anyway given the high cost of checking with the imageData, you'd much better draw twice than check.

Rq1 : I wonder to which extent using Xor mode + testing against 0 32 bits at a time could boost the compare time. (getImageData is slow.)

Rq2 : I'm sure there's a hacky way to use the playback rate to 'sync' the video and the display, and to know which frame is a genuine ( == not interpolated ) frame. ( so two pass here 1) sync 2) rewind and retrieve frames).

Rq3 : If your purpose is to get each and every video frame and only the video's frame, a browser is not the way to go. As explained above, the (desktop) browsers do interpolate to match as closely as possible the display frame rate. Those frames were not in the original stream. There are even some high-end '3D' (2D+time) interpolation devices where the initial frames are not even meant to be displayed (! ). On the other hand, if you are okay with the (interpolated) output stream, polling on rAF will provide every frames that you see (you can't miss a frame (except obviously your app is busy doing something else) .

Rq4 : interpolation ( == no duplicate frames ) is 99.99% likely on recent/decent GPU powered desktop.

Rq5 : Be sure to warm your functions (call them 100 times on start) and to create no garbage to avoid jit / gc pauses.

5
  • Thanks, I appreciate the answer! So are you saying that currentTime is changed if & only if a new frame is drawn? Are you sure? If yes, then currentTime will be very, very useful to me because it's the equivalent of a "missing" onFrameChange event.
    – markE
    Feb 11, 2015 at 21:36
  • 1
    A simple numeric comparision of currentTime is tempting to go with. The project results will be skewed if I analyze the same frame twice and there is a high cost of failing to recognize a same-frame "duplicate" sampling. Similarly there is also a high cost of missing a frame as is theoretically possible when capturing frames in a running sequence using rAF. Quite frankly, these possibilities worry me. So given the lack of a definitive answer, I'll have to pull each video into Premier so I'm sure about the frames. Still, thanks--I'm accepting your answer and upvoting.
    – markE
    Feb 12, 2015 at 2:38
  • 2
    If i understand right your requirements, you should use a software, unless you succeed to use a trick. See my comments above... And now going to bed... as a 10K ! ;-) Feb 12, 2015 at 3:11
  • Congratulations on achieving 10K...Nice!
    – markE
    Feb 12, 2015 at 3:46
  • 4
    This doesn't seem to work at a recent chrome, as it displays the interval time. currentTime seems to interpolate between the frames as it changes in each interval step Jun 12, 2019 at 11:46
3

I've got a good answer here, it uses requestVideoFrameCallback and should help you out:

How do I get the frame rate of an HTML video with JavaScript?

Explanations are at the link above ^

    // Part 1:
    var vid = document.querySelector("video");
    var last_media_time, last_frame_num, fps;
    var fps_rounder = [];
    var frame_not_seeked = true;
    // Part 2 (with some modifications):
    function ticker(useless, metadata) {
      var media_time_diff = Math.abs(metadata.mediaTime - last_media_time);
      var frame_num_diff = Math.abs(metadata.presentedFrames - last_frame_num);
      var diff = media_time_diff / frame_num_diff;
      if (
        diff &&
        diff < 1 &&
        frame_not_seeked &&
        fps_rounder.length < 50 &&
        vid.playbackRate === 1 &&
        document.hasFocus()
      ) {
        fps_rounder.push(diff);
        fps = Math.round(1 / get_fps_average());
        document.querySelector("p").textContent = "FPS: " + fps + ", certainty: " + fps_rounder.length * 2 + "%";
      }
      frame_not_seeked = true;
      last_media_time = metadata.mediaTime;
      last_frame_num = metadata.presentedFrames;
      vid.requestVideoFrameCallback(ticker);
    }
    vid.requestVideoFrameCallback(ticker);
    // Part 3:
    vid.addEventListener("seeked", function () {
      fps_rounder.pop();
      frame_not_seeked = false;
    });
    // Part 4:
    function get_fps_average() {
      return fps_rounder.reduce((a, b) => a + b) / fps_rounder.length;
    }
<p>The FPS will appear here!</p>
<video id="myVideo" width="320" height="176" controls>
  <source src="https://www.w3schools.com/tags/mov_bbb.mp4" type="video/mp4">
</video>

0

here is the code I use for calculating fps of a video. It does have error sometimes.

function getframerat(video_elem){
  document.getElementsByClassName("vid-info")[0].textContent = "Calculating..."
  //first timeout is your choice I did it for accuracy, you can replace this timeout with any other conditional checks. 
  //Just make sure video is playing, that's the important part.
  setTimeout(function(){
    let getframerates = video_elem.getVideoPlaybackQuality().totalVideoFrames
    setTimeout(function(){
      getframerates = video_elem.getVideoPlaybackQuality().totalVideoFrames - getframerates;
      //assuming frame rates are always even.
      if(getframerates%2 != 0) getframerat()
      else document.getElementsByClassName("vid-info")[0].textContent = getframerates;
    },1000)
  },1000)
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.