7

To output a single frame from ffmpeg I can do:

ffmpeg -i input.flv -ss 00:00:14.435 -vframes 1 out.png

And to output an image every second, I can do:

ffmpeg -i input.flv -vf fps=1 out%d.png

Would there be a way to create a thumbnail sprite from these outputs to help with the creation of a vtt file for thumbs on seek position?

1
  • I don't have the rep to comment on the above comments, but here is some background on what the OP means: support.jwplayer.com/customer/portal/articles/1407439 is a description of how WebVTT files are consumed by players, in this case JWPlayer, to allow what is referred to as "Preview Thumbnails" (images that hover over a progress bar, in their version), or in the "Sprite Thumbnails" example, coordinates are used with a large image that is composed of smaller images.
    – Adam Cox
    Commented Oct 27, 2017 at 17:56

4 Answers 4

9

Here is an example to create jpeg files (280x180) from mp4 video, using ffmpeg, and then assembling this thumbnails into a sprite (png format) using PHP gd2 + writing a VTT file for the video player.

First create 1 image per second with ffmpeg:

ffmpeg -i sprite/MyVideoFile.mp4 -r 1 -s 280x180 -f image2 sprite/thumbs/thumb-%%d.jpg

Then create sprite file + vtt file (example with PHP):

$dirToScan      =   'thumbs/';
$filePrefix     =   'thumb-';
$fileSuffix     =   '.jpg';
$thumbWidth     =   280;
$thumbHeight    =   180;
$imageFiles     =   array();
$spriteFile     =   'sprite.png';
$imageLine      =   20;
$vttFile        =   'sprite.vtt';
$dst_x          =   0;
$dst_y          =   0;


# read the directory with thumbnails, file name in array
foreach (glob($dirToScan.$filePrefix.'*'.$fileSuffix) as $filename) {
    array_push($imageFiles,$filename);
}

natsort($imageFiles);
#calculate dimension for the sprite 
        $spriteWidth =  $thumbWidth*$imageLine;
        $spriteHeight   =   $thumbHeight*(floor(count($imageFiles)/$imageLine)+1);

        # create png file for sprite
        $png = imagecreatetruecolor($spriteWidth,$spriteHeight);

        # open vtt file
            $handle =   fopen($vttFile,'wb+');
            fwrite($handle,'WEBVTT'."\n");

        # insert thumbs in sprite and write the vtt file
            foreach($imageFiles AS $file)   {
                $counter        =   str_replace($filePrefix,'',str_replace($fileSuffix,'',str_replace($dirToScan,'',$file)));
                $imageSrc = imagecreatefromjpeg($file);
                imagecopyresized ($png, $imageSrc, $dst_x , $dst_y , 0, 0, $thumbWidth, $thumbHeight, $thumbWidth,$thumbHeight);

                $varTCstart =   gmdate("H:i:s", $counter-1).'.000';
                $varTCend   =   gmdate("H:i:s", $counter).'.000';

                $varSprite  =   $spriteFile.'#xywh='.$dst_x.','.$dst_y.','.$thumbWidth.','.$thumbHeight;

                fwrite($handle,$counter."\n".$varTCstart.' --> '.$varTCend."\n".$varSprite."\n\n");

create new line after 20 images
                if ($counter % $imageLine == 0) {
                    $dst_x=0;
                    $dst_y+=$thumbHeight;
                }
                else    {
                    $dst_x+=$thumbWidth;
                }

            }
            imagepng($png,$spriteFile);
            fclose($handle);

VTT file looks like:

WEBVTT
1
00:00:00.000 --> 00:00:01.000
sprite.png#xywh=0,0,280,180

2
00:00:01.000 --> 00:00:02.000
sprite.png#xywh=280,0,280,180

3
00:00:02.000 --> 00:00:03.000
sprite.png#xywh=560,0,280,180
...
1
  • Nice! Just pointing out the $spriteHeight can probably be simplified a bit with: $spriteHeight = $thumbHeight*(ceil(count($imageFiles)/$imageLine)); This also gets rid of the extra row if we something that divides evenly like 40 images and 20 per row. Commented May 19, 2023 at 14:43
6

I am not completely sure what you need/mean with sprite for a vtt file, but there is a nice tool that allows you to combine single images into a big overview picture:

ImageMagick comes with a handy tool called montage

montage - create a composite image by combining several separate images. The images are tiled on the composite image optionally adorned with a border, frame, image name, and more.

You can put the thumbnails together on one or more images with the following command:

montage *.png -tile 4x4 overview.png

It will automatically generate the number of needed pictures to give you an overview.

4
  • Please let me know if this is what you need. If not I will delete this answer.
    – Mark
    Commented Dec 30, 2015 at 13:52
  • 3
    VTT (or WebVTT) can refer to the time and location of the desired screenshot like this: JW Player Adding Preview Thumbnails. FFmpeg can make a filmstrip/montage (example 1, example 2), and can make VTT files, but does not yet add location and time.
    – llogan
    Commented Dec 30, 2015 at 17:51
  • @LordNeckbeard you should add your comment as answer. I think I was completely on the wrong track, regarding the question.
    – Mark
    Commented Dec 30, 2015 at 19:19
  • You provided a solution to the montage part at least.
    – llogan
    Commented Dec 30, 2015 at 19:30
2

A little improvement proposal to the answer given by MR_1204: if the final sprite is a PNG, I would avoid the quality decrease caused by JPG compression and rather save the temporary thumbnails as PNGs:

ffmpeg -i sprite/MyVideoFile.mp4 -r 1 -s 280x180 -f image2 sprite/thumbs/thumb-%%d.png

Also, saving to PNG is usually a (tiny) little bit faster than saving to JPG.

Instead, to keep the download small, it might make sense to save (only) the final sprite image as JPG, which in case of PHP and GD only requires to replace imagepng(...) with imagejpeg(...).

0

Python solution of MR_1204's solution

import ffmpeg
import logging
from pathlib import Path
from config import temp
import os
from PIL import Image
from datetime import datetime, timedelta
import math
logger = logging.getLogger(__name__)


class ScreenshotExtractor:
    def __init__(self):
        self.screenshot_folder = f"{temp}ss"
        self.gap = 10
        self.size = "177x100"
        self.col = 5
        self.ss_width = 177
        self.ss_height = 100

    def create_sprite_sheet(self, name):
        path, dirs, files = next(os.walk(self.screenshot_folder))
        file_count = len(files)
        img_width = self.ss_width * self.col
        img_height = self.ss_height * math.ceil(file_count/self.col)

        file_name = f"{name}.jpg"
        out_image = Image.new('RGB', (img_width, img_height))
        webvtt = "WEBVTT\n\n"
        for count, img_file in enumerate(files):
            img = Image.open(f"{self.screenshot_folder}/{img_file}")
            st_time = timedelta(seconds=count * self.gap)
            end_time = timedelta(seconds=(count + 1) * self.gap)

            # Adding img to out_file
            x_mod = int(count / self.col)
            x = 0 + (count - (self.col * x_mod)) * self.ss_width
            y = 0 + x_mod * self.ss_height
            out_image.paste(img, (x, y))

            sprite = f"{file_name}#xywh={x},{y},{self.ss_width},{self.ss_height}"
            webvtt += f"{count + 1}\n0{str(st_time)}.000 --> 0{str(end_time)}.000\n{sprite}\n\n"

        out_image.save(f"{temp}{file_name}", quality=90)
        f = open(f"{temp}{name}.vtt", "w")
        f.write(webvtt)
        f.close()
        return True

    def extract_screenshots(self, file_uri, name):
        try:
            Path(self.screenshot_folder).mkdir(parents=True, exist_ok=True)

            vod = ffmpeg.input(file_uri)
            vod.output(f"{self.screenshot_folder}/{name}_%04d.png", r=1/self.gap, s=self.size).run()
            return True
        except Exception as e:
            logger.warning(e)
            return False

    def run(self, file_uri, name):
        # TODO - Actually do logic here
        self.extract_screenshots(file_uri, name)
        self.create_sprite_sheet(name)

Hope this helps somebody

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.