1

I've been using the FFmpeg libraries to read and write media files using the C API.

So far, reading seems to be pretty straightforward. I am able to read frames which I can then process, convert to RGB, process, and then convert back to YUV420 to be encoded.

The encoded files play back with VLC media player fine, and Windows Media Player if I have a codec pack installed. However, they do behave strangely: the stock Windows 10 player won't play them, same for Adobe Premiere. Also thumbnailers don't work on it. Basically it seems like nothing other than VLC or FFmpeg itself can play/process the file. I have seen this with both MP4 and MKV, so it is not a format-specific issue.

The problems go away once you remux the file with FFmpeg, for example "ffmpeg -i input.mkv -c copy output.mkv". Everything can play the file correctly. Also, the "remuxing.c" sample from the official samples works as well, with the same library version and compilers that I'm using (Visual Studio 2017, FFmpeg compiled with MinGW). It will fix the file and make it playable in all software.

I'm not sure what could be causing this. I also don't understand what the remuxing "fixed". It must be a container issue as the frames aren't touched by remuxing. I have analysed the output MKVs with FFprobe -show_packets. It seems to have budged the packet timestamps a little constant factor, and the output stream now has is_avc=true and nal_length_size=4 instead of is_avc=false and nal_length_size=0, but apart from that the files are identical.

Now here's the output of FFprobe with the 3 last test packets, stream info and format info for both streams. As you can see, they are identical except for a couple of field. But something in here must have been "fixed" during remuxing to make it work.

[PACKET]
codec_type=video
stream_index=0
pts=59050
pts_time=59.050000
dts=58890
dts_time=58.890000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=427
pos=277358
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=58970
pts_time=58.970000
dts=58970
dts_time=58.970000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=205
pos=277792
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=59130
pts_time=59.130000
dts=59050
dts_time=59.050000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=268
pos=278004
flags=__
[/PACKET]
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=Main
codec_type=video
codec_time_base=1/2000
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=720
height=576
coded_width=720
coded_height=576
has_b_frames=2
sample_aspect_ratio=N/A
display_aspect_ratio=N/A
pix_fmt=yuv420p
level=50
color_range=unknown
color_space=unknown
color_transfer=unknown
color_primaries=unknown
chroma_location=left
field_order=progressive
timecode=N/A
refs=1
is_avc=false
nal_length_size=0
id=N/A
r_frame_rate=299/12
avg_frame_rate=1000/1
time_base=1/1000
start_pts=0
start_time=0.000000
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=737
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
TAG:DURATION=00:00:59.211000000
[/STREAM]
[FORMAT]
filename=testEncLeft.mkv
nb_streams=1
nb_programs=0
format_name=matroska,webm
format_long_name=Matroska / WebM
start_time=0.000000
duration=59.211000
size=278349
bit_rate=37607
probe_score=100
TAG:COMMENT=Slickline Player Export
TAG:ENCODER=Lavf57.83.100
[/FORMAT]

And the info after remuxing, which works:

[PACKET]
codec_type=video
stream_index=0
pts=59050
pts_time=59.050000
dts=58890
dts_time=58.890000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=427
pos=277418
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=58970
pts_time=58.970000
dts=58970
dts_time=58.970000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=205
pos=277852
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=59130
pts_time=59.130000
dts=59050
dts_time=59.050000
duration=1
duration_time=0.001000
convergence_duration=N/A
convergence_duration_time=N/A
size=268
pos=278064
flags=__
[/PACKET]
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=Main
codec_type=video
codec_time_base=1/2000
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=720
height=576
coded_width=720
coded_height=576
has_b_frames=2
sample_aspect_ratio=N/A
display_aspect_ratio=N/A
pix_fmt=yuv420p
level=50
color_range=unknown
color_space=unknown
color_transfer=unknown
color_primaries=unknown
chroma_location=left
field_order=progressive
timecode=N/A
refs=1
is_avc=true
nal_length_size=4
id=N/A
r_frame_rate=299/12
avg_frame_rate=1000/1
time_base=1/1000
start_pts=0
start_time=0.000000
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=737
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
TAG:DURATION=00:00:59.212000000
[/STREAM]
[FORMAT]
filename=fixedLeft.mkv
nb_streams=1
nb_programs=0
format_name=matroska,webm
format_long_name=Matroska / WebM
start_time=0.000000
duration=59.212000
size=278409
bit_rate=37615
probe_score=100
TAG:COMMENT=Slickline Player Export
TAG:ENCODER=Lavf58.12.100
[/FORMAT]

Here is how I'm setting up the output context, for reference: it's pretty standard, following the sample code.

int ret;

avformat_alloc_output_context2(&outputFormatCtx, nullptr, nullptr, outFilePath.c_str());

av_dict_set(&outputFormatCtx->metadata, "comment", "FFmpeg Export", 0);

if (!outputFormatCtx)
{
    LOG_AND_THROW("Could not allocate output context");
}

outputVideoStream = avformat_new_stream(outputFormatCtx, nullptr);
outputVideoStream->time_base = AVRational{ 1, AV_TIME_BASE }; // Stream timebase will be used by codec

if (!outputVideoStream)
{
    LOG_AND_THROW("Failed allocating output stream");
}

// defaults to "libx264"
AVCodec *outCodec = avcodec_find_encoder_by_name(selectedCodecName.c_str());

if (!outCodec)
{
    LOG_AND_THROW("Failed finding output codec");
}

AVDictionary *opts = nullptr;

if (selectedCodecName == "libx264")
{
    opts = getX264CodecOptions();
}

encoderCtx = avcodec_alloc_context3(outCodec);

if (!encoderCtx)
{
    LOG_AND_THROW("Failed to allocate the encoder context");
}

encoderCtx->width = width;
encoderCtx->height = height;
encoderCtx->pix_fmt = AV_PIX_FMT_YUV420P;
// time base for the frames we will provide to the encoder
encoderCtx->time_base = AVRational{ 1, AV_TIME_BASE };
// convert framerate from double to rational
encoderCtx->framerate = AVRational{ (int)(frameRate * AV_TIME_BASE), AV_TIME_BASE};

// Match encoderCtx time base for the stream
outputVideoStream->time_base = encoderCtx->time_base;

ret = avcodec_open2(encoderCtx, outCodec, &opts);

if (ret < 0)
{
    LOG_AND_THROW_PARAM("Cannot open video encoder for stream: %d", ret);
}

// Fill in some params for MP4 stream, details about encoder
ret = avcodec_parameters_from_context(outputVideoStream->codecpar, encoderCtx);

if (ret < 0)
{
    LOG_AND_THROW_PARAM("Failed to copy encoder parameters to output stream: %d", ret);
}

if (outputFormatCtx->oformat->flags & AVFMT_GLOBALHEADER)
{
    encoderCtx->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
}

av_dump_format(outputFormatCtx, 0, filePath.c_str(), 1);

// End of encoder settings, setting up MP4
if (!(outputFormatCtx->oformat->flags & AVFMT_NOFILE))
{
    ret = avio_open(&outputFormatCtx->pb, outFilePath.c_str(), AVIO_FLAG_WRITE);

    if (ret < 0)
    {
        LOG_AND_THROW_PARAMSTR("Could not open output file '%s'", outFilePath.c_str());
    }
}

ret = avformat_write_header(outputFormatCtx, nullptr);

if (ret < 0)
{
    LOG_AND_THROW_PARAM("Error occurred when opening output file for writing: %d", ret);
}

Can anyone help me figure out why the container is not playing properly?

Thanks in advance.

-James

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.