3

Question: What does the Libav/FFmpeg decoding pipeline need in order to produce valid presentation timestamps (PTS) in the decoded AVFrames?

I'm decoding an H264 stream received via RTSP. I use Live555 to parse H264 and feed the stream to my LibAV decoder. Decoding and displaying is working fine, except I'm not using timestamp info and get some stuttering.

After getting a frame with avcodec_decode_video2, the presentation timestamp (PTS) is not set.

I need the PTS in order to find out for how long each frame needs to be displayed, and avoid any stuttering.

Notes on my pipeline

  • I get the SPS/PPS information via Live555, I copy these values to my AVCodecContext->extradata.
  • I also send the SPS and PPS to my decoder as NAL units, with the appended {0,0,0,1} startcode.
  • Live555 provides presentation timestamps for each packet, these are in most cases not monotonically increasing. The stream contains B-frames.
  • My AVCodecContext->time_base is not valid, value is 0/2.

Unclear:

  • Where exactly should I set the NAL PTS coming from my H264 sink (Live555)? As the AVPacket->dts, pts, none, or both?
  • Why is my time_basevalue not valid? Where is this information?
  • According to the RTP payload spec. It seems that

The RTP timestamp is set to the sampling timestamp of the content. A 90 kHz clock rate MUST be used.

  • Does this mean that I must always asume a 1/90000 timebase for the decoder? What if some other value is specified in the SPS?

2 Answers 2

1
+50

Copy the live555 pts into the avpacket pts. Process the packet with avcodec_decode_video2, and then retrieve the pts from avframe->pkt_pts, these will be monotonically increasing.

There is no need to set anything in the codec context, apart from setting the SPS and PPS in the AVCodecContex extradata

You can find a good example in VLC's github: Setting AVPacket pts: https://github.com/videolan/vlc/blob/master/modules/codec/avcodec/video.c#L983

Decoding AVPacket into AVFrame: https://github.com/videolan/vlc/blob/master/modules/codec/avcodec/video.c#L1014

Retrieving from AVFrame pts: https://github.com/videolan/vlc/blob/master/modules/codec/avcodec/video.c#L1078

Sign up to request clarification or add additional context in comments.

Comments

0

avcodec_decode_video2() reorders the frames so that decode order and presentation order is the same. Even if you somehow convince ffmpeg to give you PTS on the decoded frame it should be the same as DTS.

//
// decode a video frame
//

avcodec_decode_video2
(
    ctxt->video_st->codec,
    frame,
    &is_finished,
    buffer
);

if (buffer->dts != AV_NOPTS_VALUE)
{
    //
    // you should end up here
    //
    pts = buffer->dts;
}
else
{
    pts = 0;
}

//
// adjust time base
//
pts *= av_q2d(ctxt->video_st->time_base);

2 Comments

What if my context's time_base is not valid? As mentioned in the question its value is 0/2
Also, the buffer-dts value is AV_NOPTS_VALUE in my case.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.