78

I'm looking for a way to view HTML5 <video>, frame-by-frame.

The scenario: Having a video, with an additional button that skips to the next frame when pressed.

What do you think is the best method to do this?

  1. Play the video normally, Listen to timeUpdate event, which on FireFox is called for every frame, and then just pause the video. However, the other browsers don't behave like Firefox.
  2. Change the currentTime element manually to +1/24 of a second, where "24" is the frame rate. I have no idea how to aquire the FPS, however.
  3. Any other helpful way you can think of.

EDIT

I have found this very useful HTML5 test page, which tracks all browsers' ability to achieve accurate frame-seeking.

9
  • IIRC it's still debated in WHATWG, and there's no good solution yet.
    – Kornel
    Nov 28, 2010 at 18:18
  • @Ory Band This is exactly what I'm playing with at the moment. Have you had any luck at all? Aug 24, 2013 at 2:29
  • 7
    Test page returns a 404, any chance that someone saved the content?
    – Germain
    Feb 1, 2015 at 11:39
  • 1
    @Germain: inconduit.com/smpte/. The following link may also be relevant: erasche.github.io/videojs-framebyframe. Nov 9, 2015 at 5:02
  • 1
    @JoãoPereira sorry, but I haven't dealt with this issue in a long time. I remember that if you divide by a frame rate ratio which is far bigger than what you need might solve the problem and give you a good precision. However this is just on top of my head.
    – Ory Band
    Dec 5, 2016 at 15:28

5 Answers 5

36

It seems that most browsers allow the second approach, although you would need to know the frame rate. Opera, however, is the exception, and requires an approach similar to your first one (the result is not perfect). Here's a demo page I came up with that uses a 29.97 frames/s video (U.S. television standard). Note that it has not been extensively tested, so it might not work in IE 9, Firefox 4, or future versions of any browser.

HTML:

<p id="time"></p>
<video id="v0" controls tabindex="0" autobuffer preload>
    <source type="video/webm; codecs=&quot;vp8, vorbis&quot;" src="http://www.html5rocks.com/tutorials/video/basics/Chrome_ImF.webm"></source>
    <source type="video/ogg; codecs=&quot;theora, vorbis&quot;" src="http://www.html5rocks.com/tutorials/video/basics/Chrome_ImF.ogv"></source>
    <source type="video/mp4; codecs=&quot;avc1.42E01E, mp4a.40.2&quot;" src="http://www.html5rocks.com/tutorials/video/basics/Chrome_ImF.mp4"></source>
    <p>Sorry, your browser does not support the &lt;video&gt; element.</p>
</video>

JavaScript (run on page load and uses jQuery 1.4.4 for the sake of brevity):

var vid = $('#v0')[0];

vid.onplay = vid.onclick = function() {
    vid.onplay = vid.onclick = null;
    
    setTimeout(function() {
        vid.pause();
        setInterval(function() {
            if($.browser.opera) {
                var oldHandler = vid.onplay;
                vid.onplay = function() {
                    vid.pause();
                    vid.onplay = oldHandler;
                };
                vid.play();
            } else {
                vid.currentTime += (1 / 29.97);
            }
        }, 2000);
    }, 12000);
    
    setInterval(function() {
        $('#time').html((vid.currentTime * 29.97).toPrecision(5));
    }, 100);
};
3
  • 3
    Just an FYI, var something = things = 'stuff'; actually makes "things" global. in this case it works, in others it might not
    – newshorts
    Jul 19, 2015 at 1:46
  • Is it valid for videojs player?
    – EdG
    Jan 22, 2017 at 17:29
  • 1
    I don't think this would work as frametime isn't constant. certain frames are fast, certain are slow. you can verify by reading back each frame's currentTime
    – Bill Yan
    Jun 28, 2019 at 20:52
15

Having just been fighting this very same problem I cam up with a brute force method to find the frame rate.Knowing that the frame rate will never pass 60 frames a second I seek through the video at 1/60th second steps. Comparing each frame to the last by putting the video onto a canvas element and then using getImageData to get pixel data. I do this over a second of video and then count up the total number of unique frames to get the frame rate.

You do not have to check every single pixel. I Grab about 2/3rds of the inside of the video and then check about every 8th pixel across and down (depending on the size). You can stop the compare with just a single pixel different so this method in reasonably fast and reliable, well not compared to say reading a frameRate property, but better than guessing.

8
  • 1
    What do you use to seek and compare? Nov 3, 2014 at 15:57
  • 2
    I start the seek with video.currentTime = time; and then time = time + 1/60; In the video.seeked event I use setTimeout(frameCompare.bind(videoHandler),0); to compare the frames and then start the next seek if time < 1
    – Blindman67
    Nov 7, 2014 at 2:43
  • can it be extended to take into account us tv frame rates of 29.97 or does it just fudge the numbers? Would the additional maths required to get frame rates to that accuracy be a large increase in processing power? Nov 10, 2014 at 16:51
  • Sorry for late reply. The 29.97 frame rate will report 29 frames in one second. You can use that to deduce the 29.97.
    – Blindman67
    May 18, 2015 at 1:40
  • 2
    What if the video has a blank starting section, or any random section that doesn't change for a second?
    – Cocoa Nub
    Dec 29, 2015 at 22:22
6

The best thing you can probably do until standards are confirmed and implemented (that'll be ages!) is take a guess at the frame-rate and increment by that time. Unless you're in a controlled environment, I'd advise against relying heavily on HTML5... as much as I love its features, it won't be reliably supported.

1
  • 5
    But that's exactly what I'm trying to do! Do it with HTML5 the web-appy way! :) That's where the challange IS.
    – Ory Band
    Dec 26, 2010 at 13:39
6

This is my little keyPress handler I use to seek videos that don't have controls.
If the video is paused it skips by frames instead.

const video = document.querySelector('#yourVideo')
const expectedFramerate = 60 // yourVideo's framerate

function handleKey(ev) {
  let d = 0;

  switch (ev.key) {
    case ",": d = -5; break;  // normal
    case ".": d = +5; break;
    case "?": d = -10; break; // shift
    case ":": d = +10; break;
    case "<": d = -2; break;  // rightAlt
    case ">": d = +2; break;
    case " ": togglePlayback(); break;
  }

  if (d) {
    if (video.paused) video.currentTime += Math.sign(d) * 1/expectedFramerate
    else video.currentTime += d
  }
}

document.onkeypress = handleKey

function togglePlayback() {
  video.paused
    ? video.play()
    : video.pause()
}

Note, you could easily add a keybinding to increase/decrease/switch the expectedFramerate on the fly to suit your needs.


Bonus: Add the script as Bookmarklet

bookmarklet

https://gist.github.com/ackvf/b180e9883069ad753969a30cb2622787

2
  • 1
    I think the 1/60 is to allow for 60 frames per second? If so, might be good to highlight that users will likely want to adjust to their own frame rate, assuming they know it and it is constant.
    – Mick
    Mar 4, 2021 at 21:49
  • @Mick Good point, thank you for that comment, now, be a good pal and upvote my answer 😁
    – Qwerty
    Mar 5, 2021 at 15:37
4
+100

Check this:

https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/seekToNextFrame

It's experimental at this moment and shouldn't be used on production site but looks like up-to-date solution.

It works for Firefox 56 for now

var seekCompletePromise = HTMLMediaElement.seekToNextFrame();

HTMLMediaElement.seekToNextFrame();
1
  • 1
    This non-standard method has been deprecated by Firefox and was never implemented by any other browser vendor. Jan 3 at 9:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.