1

In photoshop and affinity photo there is a nice overlay filter that basically subtracts the overlaying layer from the one bellow to make a "difference map"

Example: Base image: Screenshot Mario Kart 1

Overlaying image: Screenshot Mario Kart 2

Resulting output: Diff from two screenshots

This is an example diff of overlaying a PNG with a JpegXL(I think) compressed image and then brightened: Diff generated from image compression

Location of filter in photoshop: Photoshop screenshot

Is there anyway to accomplish this with a filter in ffmpeg for an entire video?

2
  • What do you want to difference the video frames with? Do you have a static reference image or something? Jun 10, 2022 at 8:07
  • @MarkSetchell I want to generate a difference for each frame between two videos. I basically want to make an animated version of the compression difference image to visualise video compression in an interesting way. So have the base be a lossless video and the overlay be an x264 video for example Jun 10, 2022 at 8:10

2 Answers 2

1

Okay I found what is probbaly the best way to do it. there is an ffmpeg filter called blend which has a few modes which can work for this.

ffmpeg -r VIDEOFPS -i VIDEO -r VIDEOFPS -i VIDEO -lavfi blend=all_mode=grainextract -c:v libx264 -crf 0 -an ./out.mp4

grainextract will produce a grey video with only the compression artifacts, this seems to be made directly for this purpose.

ffmpeg -r VIDEOFPS -i VIDEO -r VIDEOFPS -i VIDEO -lavfi blend=all_mode=subtract -c:v libx264 -crf 0 -an ./out.mp4

Subtract mode will produce a green video for some reason but it will also only have the video compression.

ffmpeg -r VIDEOFPS -i VIDEO -r VIDEOFPS -i VIDEO -lavfi blend=all_mode=difference -c:v libx264 -crf 0 -an ./out.mp4

differenmce seems to be extremely close to subtract if not the exact same output.

ffmpeg -r VIDEOFPS -i VIDEO -r VIDEOFPS -i VIDEO -lavfi blend=all_mode=xor -c:v libx264 -crf 0 -an ./out.mp4

xor results in a much noiser and contrasting output but it's great for seeing exact compression artifacts in darker or less contrasting scenes

ffmpeg -r VIDEOFPS -i VIDEO -r VIDEOFPS -i VIDEO -lavfi blend=all_mode=phoenix -c:v libx264 -crf 0 -an ./out.mp4

I treid phoenix on a whim just because it sounded cool and it actually works for this usecase, it results in a pink video with only compression artifacts visible but a lot less defined then grainextract so it might be more accurate? idk

Note: if you are using any recordings from OBS or Nvidia ShadowPlay you MUST specify the FPS when importing, both use variable fps to handle small hiccups and fps differences so then using the filter you will get a bunch of random 1 frame desync

0

A non ffmpeg option:

I just remembered that blender exists and has some lovely compositing nodes.

In blender just open the compositing tab, import the two videos using the movie node and then subtract them from eachother using the math node. In a movie clip editor window you can see how many frames the video has so you can change the output length in the timeline.

Screenshot of blender's comositing setup

You could further refine the output using more math nodes and with compositing becoming real time this could be very useful for seeing the output before actually saving it as a video!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.