1

I'm trying to read a video file and store the frames as a series of images. I'm using VideoReader but for some reason I'm having trouble. I want to store the frames of two videos encoded differently and measure the structural similarity and PSNR between the two on a frame-by-frame basis.

Essentially I've three video files, an original (which reads fine), one compressed with VP9 using ffmpeg, and one compressed with H.624 using ffmpeg. The original video was originally just a set of frames merged into a .avi video using VirtualDub. The compressed videos are also .avi container.

The VP9 video appeared to work fine but when I open the images using imshow() they appear to be just a solid green color. The video opens fine on VLC so I'm not sure what the problem is.

The H.264 video doesn't read at all. When it attempts to enter the "while hasFrame()" loop, it skips over it which leads to believe Matlab thinks the video frames aren't there? Again, this video opens fine in VLC and the three videos look almost identical.

Anyone got any ideas why this is happening? Is it to do with the way Matlab decodes the video or some parameters set by ffmpeg?

Original vs VP9 in Matlab

Console output for ffmpeg - VP9

Console output for ffmpeg - H264

Main file:

test_vid = 'vp9.avi';
images = readVideos(test_vid);

for i=1:length(images)

  % Convert from cells to matrices
  image1 = cell2mat(images(1,i));
  image2 = cell2mat(images(2,i));

  % Do stuff with the images here

  subplot(1,2,1);
  imshow(image1);
  subplot(1,2,2);
  imshow(image2);
end

ReadVideos():

function images = readVideos(test_video)

    % Video directories
    test_video_dir = strcat('src/', test_video);
    v_original = VideoReader('src/input.avi');
    v_test = VideoReader(test_video_dir);

    % Read original video
    i = 1;
    v_original.CurrentTime = 5;
    while hasFrame(v_original)
        frame = readFrame(v_original);
        originalImages{i} = frame;
        i = i + 1;
    end

    % Read test video
    i = 1;
    v_test.CurrentTime = 5;
    while hasFrame(v_test)
         frame = readFrame(v_test);
         testImages{i} = frame;
         i = i + 1;
    end

    images = cat(1, originalImages, testImages);
end

On a side note, is Matlab the best choice for the task or is there specialised software out there for doing this?

1 Answer 1

1

Matlab's Videoreader relies, for the most part, on the decoders provided by the OS. In general, these decoders have limited support for streams with pixel formats other than the most common ones. That appears to be the case here.

Your VP9 output is gbrp i.e. planar (8-bit) RGB. Your H.264 output is YUV but with 4:4:4 chroma sampling. Both of these are non-standard.

To read these files with Videoreader, convert to a standard pixel format:

ffmpeg -i input.avi -pix_fmt yuv420p -c:v libvpx-vp9 vp9.avi

and

ffmpeg -i input.avi -pix_fmt yuv420p -c:v libx264 h264.avi

However, you may encounter issues when carrying out quality metric comparisons since they require both operands to have the same pixel format. Matlab may or may not convert them automatically.


You could just perform the quality check in ffmpeg.

ffmpeg -i input.avi -pix_fmt gbrp -c:v libvpx-vp9 vp9.mkv

and

ffmpeg -i input.avi -pix_fmt gbr24 -c:v libx264rgb h264.mkv

And then

ffmpeg -i input.avi -i vp9.mkv -i h264.mkv -filter_complex "[1][0]psnr=f=vp9-psnr.txt:stats_version=2;[1][0]ssim=f=vp9-ssim.txt;[2][0]psnr=f=h264-psnr.txt:stats_version=2;[2][0]ssim=f=h264-ssim.txt" -f null -

See http://www.ffmpeg.org/ffmpeg-filters.html#psnr and http://www.ffmpeg.org/ffmpeg-filters.html#ssim

3
  • Hi Gyan, thanks for your answer! Your solution works perfectly for the H264 however I'm still getting the same problem with the VP9, any thoughts? Also, thank you for your second suggestion, it's a great way of doing it and I'm getting results for the VP9 through that. The results from the quality check (For H264, at least) from Matlab and ffmpeg differ slightly (about 4/5% for SSIM), any thoughts on which method is more reliable?
    – Jordan
    May 2, 2019 at 14:18
  • I'm still getting the same problem with the VP9, any thoughts? --> I think's no Directshow/Windows Media Foundation decoder for VP9 installed.
    – Gyan
    May 2, 2019 at 15:06
  • Matlab will carrying out some pixel format conversion - either on the reference or the transcoded video. I would stick with the ffmpeg filter.
    – Gyan
    May 2, 2019 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.