7

I'm currently trying to edit videos with the Python library of FFMPEG. I'm working with multiple file formats, precisely .mp4, .png and text inputs (.txt). The goal is to embed the different video files within a "layout" - for demonstration purposes I tried to design an example picture:

Example

The output is supposed to be a 1920x1080 .mp4 file with the following Elements:

  • Element 3 is the video itself (due to it being a mobile phone screen recording, it's about the size displayed there)
  • Element 1 and 2 are the "boarders", i.e. static pictures (?)
  • Element 4 represents a regularly changing text - input through the python script (probably be read from a .txt file)
  • Element 5 portrays a .png, .svg or alike; in general a "picture" in the broad sense.

What I'm trying to achieve is to create a sort of template file in which I "just" need to input the different .mp4 and .png files, as well as the text and in the end I'll receive a .mp4 file whereas my Python script functions as the navigator sending the data packages to FFMPEG to process the video itself.

I dug into the FFMPEG library as well as the python-specific repository and wasn't able to find such an option. There were lots of articles explaining the usage of "channel layouts" (though these don't seem to fit my need).

In case anyone wants to try on the same versions:

  • python --version: Python 3.7.3
  • pip show ffmpeg: Version: 1.4 (it's the most recent; on an off-topic note: It's not obligatory to use FFMPEG, I'd prefer using this library though if it doesn't offer the functionality I'm looking for, I'd highly appreciate if someone suggested something else)
11
  • 1
    Would you accept ffmpeg command line example? Or do you need ffmpeg-python example specifically?
    – llogan
    Commented Jan 5, 2021 at 17:52
  • @llogan Well as my whole architecture is built upon Python, I'd appreciate a Python-based ffmpeg solution, though I'm thankful for everything I can find! To my knowledge both of the libraries don't differ as much so hopefully it's supported with Python (i.e. I would try to reproduce your approach in Python). Commented Jan 5, 2021 at 18:14
  • @llogan So I re-checked the documentation and with the help of a pure ffmpeg solution I should be able to transfer it into Python.. Nearly all the functionality is provided by the Python-based ffmpeg (all well-documented), hence I'm fairly confident that this is doable. Commented Jan 5, 2021 at 21:53
  • @llogan Might wanna give it a shot? The question is bountied now and as you're one of the only three person on whole StackOverflow to have ever achieved the gold ffmpeg badge, I'm sure you can help out! Commented Jan 9, 2021 at 16:48
  • I have some experience with both ffmpeg and python, so i may make you a sample code before the bounty expires !
    – TheEagle
    Commented Jan 12, 2021 at 22:52

2 Answers 2

6

enter image description here

ffmpeg -i left.jpg -i video.mp4 -i right.png -i logo.png -filter_complex "[0]scale=(1920-1080*($width/$height))/2:1080:force_original_aspect_ratio=increase,crop=(1920-1080*($width/$height))/2:1080[left];[1]scale=-2:1080[main];[2]scale=(1920-1080*($width/$height))/2:1080:force_original_aspect_ratio=increase,crop=(1920-1080*($width/$height))/2:1080[right];[left][main][right]hstack=inputs=3[bg];[bg][3]overlay=5:main_h-overlay_h-5:format=auto,drawtext=textfile=text.txt:reload=1:x=w-tw-10:y=h-th-10,format=yuv420p" -movflags +faststart output.mp4

What this does is scale video.mp4 so it is 1080 pixels high and autoscale the width. left.jpg and right.png are scaled to take up the remainder so the result is 1920x1080.

This is a simple example and won't work for all inputs such as if the video.mp4 autoscaled width is bigger than 1920, but you can deal with that by using arithmetic expressions.

$width and $height refer to the size of video.mp4. See Getting video dimension / resolution / width x height from ffmpeg for a Python friendly method using JSON or XML.

See documentation for scale, crop, hstack, drawtext, overlay, and format filters.

A much simpler method is to just add colored bars instead of arbitrary images. See Resizing videos with ffmpeg to fit a specific size.

Change text on the fly by atomically updating text.txt. Or use the subtitles filter instead of drawtext if you want the text to change on certain timestamps (you did not specify what you needed).

Related answers:

9
  • Thanks for your reply! May you elaborate the implementation in Python a bit? As far as I understood this is just for the FFMPEG CLI, hence Python doesn't function as a sort of navigator or broadly speaking, your solution doesn't include e.g. changing text elements, pictures etc. Commented Jan 12, 2021 at 14:49
  • @J.M.Arnold I'm not a Python user, that's why I asked if a ffmpeg cli solution would be acceptable before providing an answer.
    – llogan
    Commented Jan 12, 2021 at 17:05
  • Regarding your question above, the side bars would be both consisting of pictures. Commented Jan 12, 2021 at 17:31
  • @J.M.Arnold 2 different pictures? Or 1 picture (same on each side)? Or 1 picture as background?
    – llogan
    Commented Jan 12, 2021 at 17:32
  • Depends, would prefer a solution for two different pictures. Commented Jan 12, 2021 at 18:31
3
+50

I've done it. The code can be used as a command line program or as a module. To find out more about the command line usage, call it with the --help option. For module usage, import the make_video fucntion in your code (or copy-paste it), and pass the appropriate arguments to it. I have included a screenshot of what my script produced with some sample material, and of course, the code. Code:

#!/usr/bin/python3
#-*-coding: utf-8-*-

import sys, argparse, ffmpeg, os

def make_video(video, left_boarder, right_boarder, picture_file, picture_pos, text_file, text_pos, output):
    videoprobe = ffmpeg.probe(video)
    videowidth, videoheight = videoprobe["streams"][0]["width"], videoprobe["streams"][0]["height"] # get width of main video
    scale = (1080 / videoheight)
    videowidth *= scale
    videoheight *= scale
    videostr = ffmpeg.input(video) # open main video
    audiostr = videostr.audio
    videostr = ffmpeg.filter(videostr, "scale", "%dx%d" %(videowidth, videoheight))
    videostr = ffmpeg.filter(videostr, "pad", 1920, 1080, "(ow-iw)/2", "(oh-ih)/2")
    videostr = videostr.split()
    boarderwidth = (1920 - videowidth) / 2 # calculate width of boarders
    left_boarderstr = ffmpeg.input(left_boarder) # open left boarder and scale it
    left_boarderstr = ffmpeg.filter(left_boarderstr, "scale", "%dx%d" % (boarderwidth, 1080))
    right_boarderstr = ffmpeg.input(right_boarder) # open right boarder
    right_boarderstr = ffmpeg.filter(right_boarderstr, "scale", "%dx%d" % (boarderwidth, 1080))
    picturewidth = boarderwidth - 100 # calculate width of picture
    pictureheight = (1080 / 3) - 100 # calculate height of picture
    picturestr = ffmpeg.input(picture_file) # open picture and scale it (there will be a padding of 100 px around it)
    picturestr = ffmpeg.filter(picturestr, "scale", "%dx%d" % (picturewidth, pictureheight))
    videostr = ffmpeg.overlay(videostr[0], left_boarderstr, x=0, y=0) # add left boarder
    videostr = ffmpeg.overlay(videostr, right_boarderstr, x=boarderwidth + videowidth, y=0) #add right boarder
    picture_y = (((1080 / 3) * 2) + 50) # calculate picture y position for bottom alignment
    if picture_pos == "top":
        picture_y = 50
    elif picture_pos == "center":
        picture_y = (1080 / 3) + 50
    videostr = ffmpeg.overlay(videostr, picturestr, x=50, y=picture_y)
    text_x = (1920 - boarderwidth) + 50
    text_y = ((1080 / 3) * 2) + 50
    if text_pos == "center":
        text_y = (1080 / 3) + 50
    elif text_pos == "top":
        text_y = 50
    videostr = ffmpeg.drawtext(videostr, textfile=text_file, reload=1, x=text_x, y=text_y, fontsize=50)
    videostr = ffmpeg.output(videostr, audiostr, output)
    ffmpeg.run(videostr)

def main():
    #create ArgumentParser and add options to it
    argp = argparse.ArgumentParser(prog="ffmpeg-template")
    argp.add_argument("--videos", help="paths to main videos (default: video.mp4)", nargs="*", default="video.mp4")
    argp.add_argument("--left-boarders", help="paths to images for left boarders (default: left_boarder.png)", nargs="*", default="left_boarder.png")
    argp.add_argument("--right-boarders", help="paths to images for right boarders (default: right_boarder.png)", nargs="*", default="right_boarder.png")
    argp.add_argument("--picture-files", nargs="*", help="paths to pictures (default: picture.png)",  default="picture.png")
    argp.add_argument("--picture-pos", help="where to put the pictures (default: bottom)", choices=["top", "center", "bottom"], default="bottom")
    argp.add_argument("--text-files", nargs="*", help="paths to files with text (default: text.txt)", default="text.txt")
    argp.add_argument("--text-pos", help="where to put the texts (default: bottom)", choices=["top", "center", "bottom"], default="bottom")
    argp.add_argument("--outputs", nargs="*", help="paths to outputfiles (default: out.mp4)", default="out.mp4")
    args = argp.parse_args()
    # if only one file was provided, put it into a list (else, later, every letter of the filename will be treated as a filename)
    if type(args.videos) == str:
        args.videos = [args.videos]
    if type(args.left_boarders) == str:
        args.left_boarders = [args.left_boarders]
    if type(args.right_boarders) == str:
        args.right_boarders = [args.right_boarders]
    if type(args.picture_files) == str:
        args.picture_files = [args.picture_files]
    if type(args.text_files) == str:
        args.text_files = [args.text_files]
    if type(args.outputs) == str:
        args.outputs = [args.outputs]

    for i in (range(0, min(len(args.videos), len(args.left_boarders), len(args.right_boarders), len(args.picture_files), len(args.text_files), len(args.outputs))) or [0]):
        print("Info : merging video %s, boarders %s %s, picture %s and textfile %s into %s" % (args.videos[i], args.left_boarders[i], args.right_boarders[i], args.picture_files[i], args.text_files[i], args.outputs[i]))
        # check if all files provided with the options exist
        if not os.path.isfile(args.videos[i]):
            print("Error : video %s was not found" % args.videos[i])
            continue
        if not os.path.isfile(args.left_boarders[i]):
            print("Error : left boarder %s was not found" % args.left_boarders[i])
            continue
        if not os.path.isfile(args.right_boarders[i]):
            print("Error : rightt boarder %s was not found" % args.right_boarders[i])
            continue
        if not os.path.isfile(args.picture_files[i]):
            print("Error : picture %s was not found" % args.picture_files[i])
            continue
        if not os.path.isfile(args.text_files[i]):
            print("Error : textfile %s was not found" % args.text_files[i])
            continue
        try:
            make_video(args.videos[i], args.left_boarders[i], args.right_boarders[i], args.picture_files[i], args.picture_pos, args.text_files[i], args.text_pos, args.outputs[i])
        except Exception as e:
            print(e)

if __name__ == "__main__":
    main()

Example for direct usage as a script:

$ ./ffmpeg-template --videos input1.mp4 inout2.mp4 --left-boarders left_boarder1.png left_boarder2.png --right-boarders right_boarder1.png right_boarder2.png --picture-files picture1.png picture2.png --text-files text1.txt text2.png --outputs out1.mp4 out2.mp4 --picture-pos bottom --text-pos bottom

As values for the options i took the defaults. If you omit the options, these defaults will be used, and if one of the files is not found, an error message will be displayed.

Image: enter image description here

13
  • @J.M.Arnold what do you mean with invalid - i can see it ...
    – TheEagle
    Commented Jan 13, 2021 at 18:28
  • Mhm okay weird bug on my side probably. Furthermore, your code - if I read correctly - only edits one video, right? It takes a video as an input, not a list of videos or as the question implies “multiple videos”, doesn’t it? Commented Jan 13, 2021 at 18:29
  • @J.M.Arnold i embedded the image into my post and edited it to make the usage more clear ;)
    – TheEagle
    Commented Jan 13, 2021 at 18:31
  • 1
    @J.M.Arnold your question title indeed says something about "multiple videos", but your question says that you have a video (element 3), two static pictures for the boarders (elements 1 and 2), a text file to be displayed that changes regularly (element 4) and a picture to be displayed (element 5), and want to overlay these as in the image you added. Thats just what my script does.
    – TheEagle
    Commented Jan 13, 2021 at 18:44
  • 1
    @J.M.Arnold i updated my answer, the code now takes as input lists of videos, boarder images, pictures, text files, and output files, and merges them as described
    – TheEagle
    Commented Jan 13, 2021 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.