2

I need to stream my desktop to the AWS MediaLive service and, as a requirement, I must include wallclock timecodes in the stream. The AWS support kindly informed me that for h.264 encoded streams, I need to provide timecodes as "pic_timing SEI messages".

I'm streaming with FFmpeg via the RTMP protocol on Windows 10 so, I tried adding the use_wallclock_as_timestamps and copyts flags to my command.

ffmpeg -f gdigrab -framerate 30 -offset_x 0 -offset_y 0 -video_size 1920x1080 -show_region 1 -use_wallclock_as_timestamps 1 -i desktop -vf scale=320:240 -c:v libx264 -c:a aac -profile:v main -level 3.1 -pix_fmt yuv420p -copyts -f flv rtmp://<ip>:1935/<app>/<stream>

However, the timecodes are not picked up by AWS MediaLive.

My questions are:

  • Does the use_wallclock_as_timestamps flag actually create timecodes as "pic_timing SEI messages" as required?
  • If not, how do I add wallclock timecodes as "pic_timing SEI messages"? It doesn't have to be every frame. Every 2 or 3 seconds would suffice.

I'm stumped. I couldn't find the answer in the FFmpeg documentation.

Thanks for your help.

2
  • The encoder (x264) has to write these. I'll check.
    – Gyan
    May 16, 2019 at 19:59
  • Thank you @Gyan, any information will be appreciated. May 17, 2019 at 10:12

2 Answers 2

3

For the "pic_timing SEI messages" support, the following encoder wrappers for H.264/AVC have this implemented in ffmpeg:

  1. h264_qsv

  2. h264_vaapi

  3. h264_nvenc

You can confirm support for the same via:

(a). For the h264_vaapi encoder wrapper (Available on Linux only):

ffmpeg -h encoder=h264_vaapi

Encoder h264_vaapi [H.264/AVC (VAAPI)]:
    General capabilities: delay hardware 
    Threading capabilities: none
    Supported hardware devices: vaapi 
    Supported pixel formats: vaapi_vld
h264_vaapi AVOptions:
  -low_power         <boolean>    E..V....... Use low-power encoding mode (only available on some platforms; may not support all encoding features) (default false)
  -idr_interval      <int>        E..V....... Distance (in I-frames) between IDR frames (from 0 to INT_MAX) (default 0)
  -b_depth           <int>        E..V....... Maximum B-frame reference depth (from 1 to INT_MAX) (default 1)
  -rc_mode           <int>        E..V....... Set rate control mode (from 0 to 6) (default auto)
     auto            0            E..V....... Choose mode automatically based on other parameters
     CQP             1            E..V....... Constant-quality
     CBR             2            E..V....... Constant-bitrate
     VBR             3            E..V....... Variable-bitrate
     ICQ             4            E..V....... Intelligent constant-quality
     QVBR            5            E..V....... Quality-defined variable-bitrate
     AVBR            6            E..V....... Average variable-bitrate
  -qp                <int>        E..V....... Constant QP (for P-frames; scaled by qfactor/qoffset for I/B) (from 0 to 52) (default 0)
  -quality           <int>        E..V....... Set encode quality (trades off against speed, higher is faster) (from -1 to INT_MAX) (default -1)
  -coder             <int>        E..V....... Entropy coder type (from 0 to 1) (default cabac)
     cavlc           0            E..V.......
     cabac           1            E..V.......
     vlc             0            E..V.......
     ac              1            E..V.......
  -aud               <boolean>    E..V....... Include AUD (default false)
  -sei               <flags>      E..V....... Set SEI to include (default identifier+timing+recovery_point)
     identifier                   E..V....... Include encoder version identifier
     timing                       E..V....... Include timing parameters (buffering_period and pic_timing)
     recovery_point               E..V....... Include recovery points where appropriate
  -profile           <int>        E..V....... Set profile (profile_idc and constraint_set*_flag) (from -99 to 65535) (default -99)
     constrained_baseline 578          E..V.......
     main            77           E..V.......
     high            100          E..V.......
  -level             <int>        E..V....... Set level (level_idc) (from -99 to 255) (default -99)
     1               10           E..V.......
     1.1             11           E..V.......
     1.2             12           E..V.......
     1.3             13           E..V.......
     2               20           E..V.......
     2.1             21           E..V.......
     2.2             22           E..V.......
     3               30           E..V.......
     3.1             31           E..V.......
     3.2             32           E..V.......
     4               40           E..V.......
     4.1             41           E..V.......
     4.2             42           E..V.......
     5               50           E..V.......
     5.1             51           E..V.......
     5.2             52           E..V.......
     6               60           E..V.......
     6.1             61           E..V.......
     6.2             62           E..V.......

For VAAPI, the private codec option -sei:v +timing must be passed to h264_vaapi.

(b). For the h264_qsv encoder wrapper:

ffmpeg -h encoder=h264_qsv

Encoder h264_qsv [H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration)]:
    General capabilities: delay hybrid 
    Threading capabilities: none
    Supported hardware devices: qsv qsv qsv 
    Supported pixel formats: nv12 p010le qsv
h264_qsv encoder AVOptions:
  -async_depth       <int>        E..V....... Maximum processing parallelism (from 1 to INT_MAX) (default 4)
  -avbr_accuracy     <int>        E..V....... Accuracy of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
  -avbr_convergence  <int>        E..V....... Convergence of the AVBR ratecontrol (from 0 to INT_MAX) (default 0)
  -preset            <int>        E..V....... (from 1 to 7) (default medium)
     veryfast        7            E..V.......
     faster          6            E..V.......
     fast            5            E..V.......
     medium          4            E..V.......
     slow            3            E..V.......
     slower          2            E..V.......
     veryslow        1            E..V.......
  -rdo               <int>        E..V....... Enable rate distortion optimization (from -1 to 1) (default -1)
  -max_frame_size    <int>        E..V....... Maximum encoded frame size in bytes (from -1 to 65535) (default -1)
  -max_slice_size    <int>        E..V....... Maximum encoded slice size in bytes (from -1 to 65535) (default -1)
  -bitrate_limit     <int>        E..V....... Toggle bitrate limitations (from -1 to 1) (default -1)
  -mbbrc             <int>        E..V....... MB level bitrate control (from -1 to 1) (default -1)
  -extbrc            <int>        E..V....... Extended bitrate control (from -1 to 1) (default -1)
  -adaptive_i        <int>        E..V....... Adaptive I-frame placement (from -1 to 1) (default -1)
  -adaptive_b        <int>        E..V....... Adaptive B-frame placement (from -1 to 1) (default -1)
  -b_strategy        <int>        E..V....... Strategy to choose between I/P/B-frames (from -1 to 1) (default -1)
  -forced_idr        <boolean>    E..V....... Forcing I frames as IDR frames (default false)
  -low_power         <boolean>    E..V....... enable low power mode(experimental: many limitations by mfx version, BRC modes, etc.) (default false)
  -cavlc             <int>        E..V....... Enable CAVLC (from 0 to 1) (default 0)
  -idr_interval      <int>        E..V....... Distance (in I-frames) between IDR frames (from 0 to INT_MAX) (default 0)
  -pic_timing_sei    <int>        E..V....... Insert picture timing SEI with pic_struct_syntax element (from 0 to 1) (default 1)
  -single_sei_nal_unit <int>        E..V....... Put all the SEI messages into one NALU (from -1 to 1) (default -1)
  -max_dec_frame_buffering <int>        E..V....... Maximum number of frames buffered in the DPB (from 0 to 65535) (default 0)
  -look_ahead        <int>        E..V....... Use VBR algorithm with look ahead (from 0 to 1) (default 0)
  -look_ahead_depth  <int>        E..V....... Depth of look ahead in number frames (from 0 to 100) (default 0)
  -look_ahead_downsampling <int>        E..V....... Downscaling factor for the frames saved for the lookahead analysis (from 0 to 3) (default unknown)
     unknown         0            E..V.......
     auto            0            E..V.......
     off             1            E..V.......
     2x              2            E..V.......
     4x              3            E..V.......
  -int_ref_type      <int>        E..V....... Intra refresh type (from -1 to 65535) (default -1)
     none            0            E..V.......
     vertical        1            E..V.......
  -int_ref_cycle_size <int>        E..V....... Number of frames in the intra refresh cycle (from -1 to 65535) (default -1)
  -int_ref_qp_delta  <int>        E..V....... QP difference for the refresh MBs (from -32768 to 32767) (default -32768)
  -recovery_point_sei <int>        E..V....... Insert recovery point SEI messages (from -1 to 1) (default -1)
  -profile           <int>        E..V....... (from 0 to INT_MAX) (default unknown)
     unknown         0            E..V.......
     baseline        66           E..V.......
     main            77           E..V.......
     high            100          E..V.......
  -a53cc             <int>        E..V....... Use A53 Closed Captions (if available) (from 0 to 1) (default 1)
  -aud               <int>        E..V....... Insert the Access Unit Delimiter NAL (from 0 to 1) (default 0)
  -mfmode            <int>        E..V....... Multi-Frame Mode (from 0 to 2) (default auto)
     off             1            E..V.......
     auto            2            E..V.......
  -repeat_pps        <boolean>    E..V....... repeat pps for every frame (default false)

The QSV encoder wrapper requires the private codec option -pic_timing_sei:v 1 to be set, and this is the default. This can be overridden to disable the same by passing 0 to the aforementioned parameter.

(c). For h264_nvenc, this option is enabled by default.

Update:

On the HEVC side, ffmpeg's hevc_nvenc wrapper supports it through the private codec option -s12m_tc option that must be set to 1 to enable. This is mentioned because of the work-in-progress support for enhanced RTMP in FFmpeg that implements HEVC.

0
1

To see if incoming stream has the data:

use ffprobe -show_packets rather than -show_frames

 "side_data_list": [
                {
                    "side_data_type": "Producer Reference Time",
                    "wallclock": 104471944699,
                    "wallclock(s)": "104471944699",
                    "flags": 24
                }

To see that data ffprobe needs a small mod:

ffprobe.c:2364

} else if (sd->type == AV_PKT_DATA_AFD && sd->size > 0) {
    print_int("active_format", *sd->data);
} else if (sd->type == AV_PKT_DATA_PRFT && sd->size > 0) {
    const AVProducerReferenceTime *prop = (AVProducerReferenceTime *)sd->data;
    print_ts("wallclock",prop->wallclock);
    print_int("flags",prop->flags);

}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.