11



I would like to generate GIF from QImage, using ffmpeg - all of that programmatically (C++). I’m working with Qt 5.6 and the last build of ffmpeg (build git-0a9e781 (2016-06-10).

I’m already able to convert these QImage in .mp4 and it works. I tried to use the same principle for the GIF, changing format pixel and codec. GIF is generated with two pictures (1 second each), in 15 FPS.

## INITIALIZATION
#####################################################################

// Filepath : "C:/Users/.../qt_temp.Jv7868.gif"  
// Allocating an AVFormatContext for an output format...
avformat_alloc_output_context2(formatContext, NULL, NULL, filepath);

...

// Adding the video streams using the default format codecs and initializing the codecs.
stream = avformat_new_stream(formatContext, *codec);

AVCodecContext * codecContext = avcodec_alloc_context3(*codec);

context->codec_id       = codecId;
context->bit_rate       = 400000;
...
context->pix_fmt        = AV_PIX_FMT_BGR8;

...

// Opening the codec...
avcodec_open2(codecContext, codec, NULL);

...

frame = allocPicture(codecContext->width, codecContext->height, codecContext->pix_fmt);
tmpFrame = allocPicture(codecContext->width, codecContext->height, AV_PIX_FMT_RGBA);

...

avformat_write_header(formatContext, NULL);

## ADDING A NEW FRAME
#####################################################################

// Getting in parameter the QImage: newFrame(const QImage & image)
const qint32 width  = image.width();
const qint32 height = image.height();

// Converting QImage into AVFrame
for (qint32 y = 0; y < height; y++) {
    const uint8_t * scanline = image.scanLine(y);

    for (qint32 x = 0; x < width * 4; x++) {
        tmpFrame->data[0][y * tmpFrame->linesize[0] + x] = scanline[x];
    }
}

...

// Scaling...
if (codec->pix_fmt != AV_PIX_FMT_BGRA) {
    if (!swsCtx) {
        swsCtx = sws_getContext(codec->width, codec->height,
                                AV_PIX_FMT_BGRA,
                                codec->width, codec->height,
                                codec->pix_fmt,
                                SWS_BICUBIC, NULL, NULL, NULL);
    }

    sws_scale(swsCtx,
              (const uint8_t * const *)tmpFrame->data,
              tmpFrame->linesize,
              0,
              codec->height,
              frame->data,
              frame->linesize);
}
frame->pts = nextPts++;

...

int gotPacket = 0;
AVPacket packet = {0};

av_init_packet(&packet);
avcodec_encode_video2(codec, &packet, frame, &gotPacket);

if (gotPacket) {
    av_packet_rescale_ts(paket, *codec->time_base, stream->time_base);
    paket->stream_index = stream->index;

    av_interleaved_write_frame(formatContext, paket);
}

But when I’m trying to modify the video codec and pixel format to match with GIF specifications, I’m facing some issues. I tried several codecs such as AV_CODEC_ID_GIF and AV_CODEC_ID_RAWVIDEO but none of them seem to work. During the initialization phase, avcodec_open2() always returns such kind of errors:

Specified pixel format rgb24 is invalid or not supported
Could not open video codec:  gif

EDIT 17/06/2016

Digging a little bit more, avcodec_open2() returns -22:

#define EINVAL          22      /* Invalid argument */

EDIT 22/06/2016

Here are the flags used to compile ffmpeg:

"FFmpeg/Libav configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --disable-w32threads --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmfx --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib"

Did I miss a crucial one for GIF?

EDIT 27/06/2016

Thanks to Gwen, I have a first output: I setted the context->pix_fmt to AV_PIX_FMT_BGR8. Btw I'm still facing some issues with the generated GIF. It's not playing and encoding appears to fail.

GIF generated in command lines with ffmpeg (left) . . . GIF generated programmatically (right)
Generated in command line with ffmpeg enter image description here

It looks like some options are not defined... also may be a wrong conversion between QImage and AVFrame? I updated the code above. It represents a lot of code, so I tried to stay short. Don't hesitate to ask more details.

End of EDIT

I’m not really familiar with ffmpeg, any kind of help would be highly appreciated. Thank you.

2
  • This shows that the codec_id field is already set by avcodec_get_context_defaults3. Did you try not assigning codecId to it?
    – Fadel
    Commented Jun 23, 2016 at 18:39
  • Thank you for your answer. Yes I did. Same result. A also tried using AVCodecContext * codecContext = avcodec_alloc_context3(*codec);.
    – Sierra
    Commented Jun 24, 2016 at 7:21

3 Answers 3

5
+25

GIF can only support 256 colors bitmaps (8 bits per pixel). This may be the reason why you have the Specified pixel format rgb24 is invalid or not supported error.

The pixel format you need to use is AV_PIX_FMT_PAL8 (8 bit with RGB32 palette).

5
  • Great. That was very hepful, thank you. I have now a first output, using AV_PIX_FMT_BGR8 (AV_PIX_FMT_PAL8 renders nothing in my case). I updated the question above.
    – Sierra
    Commented Jun 27, 2016 at 15:12
  • 1
    That's weird, your output file has 30 frames... Maybe something wrong in the way you build your paket? I can also see that your 30-frames file has one base frame, with the coloured flags, and 29 additional frames, with coloured flags too... Are you sure you're giving it the good frames ?
    – Gwen
    Commented Jun 27, 2016 at 18:28
  • GIF is generated with two pictures (1 second each), in 15 FPS. That why there are 30 frames (sorry, I would have to precise it). I edited the code above to provide more details on the build paket part. I'm testing frames...Thank you.
    – Sierra
    Commented Jun 28, 2016 at 8:46
  • Each time I add a frame, if gotPacket is true after the call of avcodec_encode_video2 function.
    – Sierra
    Commented Jun 28, 2016 at 12:16
  • Ok, but does it mean 2 or 30 times ? (I think it should be called only twice).
    – Gwen
    Commented Jun 28, 2016 at 12:45
3

Here is a way to convert QImage into GIF format, using ffmpeg. I tried to be as clear as possible, removing errors catching.

First, initializing ffmpeg :

AVOutputFormat  * outputFormat  = Q_NULLPTR;
AVFormatContext * formatContext = Q_NULLPTR;

avformat_alloc_output_context2(&formatContext, NULL, NULL, filePath.data()); // i.e. filePath="C:/Users/.../qt_temp.Jv7868.mp4" 

// Adding the video streams using the default format codecs and initializing the codecs...
outputFormat = formatContext->oformat;
if (outputFormat->video_codec != AV_CODEC_ID_NONE) {
    // Finding a registered encoder with a matching codec ID...
    *codec = avcodec_find_encoder(outputFormat->video_codec);

    // Adding a new stream to a media file...
    stream = avformat_new_stream(formatContext, *codec);
    stream->id = formatContext->nb_streams - 1;


    AVCodecContext * codecContext = avcodec_alloc_context3(*codec);

    switch ((*codec)->type) {
    case AVMEDIA_TYPE_VIDEO:
        codecContext->codec_id  = outputFormat->video_codec; // here, outputFormat->video_codec should be AV_CODEC_ID_GIF
        codecContext->bit_rate  = 400000;

        codecContext->width     = 1240;
        codecContext->height    = 874;

        codecContext->pix_fmt   = AV_PIX_FMT_RGB8;

        ...

        // Timebase: this is the fundamental unit of time (in seconds) in terms of which frame
        // timestamps are represented. For fixed-fps content, timebase should be 1/framerate
        // and timestamp increments should be identical to 1.
        stream->time_base       = (AVRational){1, fps}; // i.e. fps=1
        codecContext->time_base = stream->time_base;

        // Emit 1 intra frame every 12 frames at most
        codecContext->gop_size  = 12;
        codecContext->pix_fmt   = AV_PIX_FMT_YUV420P;

        if (codecContext->codec_id == AV_CODEC_ID_H264) {
            av_opt_set(codecContext->priv_data, "preset", "slow", 0);
        }
        break;
    }

    if (formatContext->oformat->flags & AVFMT_GLOBALHEADER) {
        codecContext->flags |= CODEC_FLAG_GLOBAL_HEADER;
    }
}

avcodec_open2(codecContext, codec, NULL);

// Here we need 3 frames. Basically, the QImage is firstly extracted in AV_PIX_FMT_BGRA.
// We need then to convert it to AV_PIX_FMT_RGB8 which is required by the .gif format.
// If we do that directly, there will be some artefacts and bad effects... to prevent that
// we convert FIRST AV_PIX_FMT_BGRA into AV_PIX_FMT_YUV420P THEN into AV_PIX_FMT_RGB8.
frame = allocPicture(codecContext->width, codecContext->height, codecContext->pix_fmt); // here, codecContext->pix_fmt should be AV_PIX_FMT_RGB8
tmpFrame = allocPicture(codecContext->width, codecContext->height, AV_PIX_FMT_BGRA);
yuvFrame = allocPicture(codecContext->width, codecContext->height, AV_PIX_FMT_YUV420P);

avcodec_parameters_from_context(stream->codecpar, codecContext);

av_dump_format(formatContext, 0, filePath.data(), 1);

if (!(outputFormat->flags & AVFMT_NOFILE)) {
    avio_open(&formatContext->pb, filePath.data(), AVIO_FLAG_WRITE);
}

// Writing the stream header, if any...
avformat_write_header(formatContext, NULL);

Then the main part, adding a QImage (received from a loop for example):

// -> parameter: QImage image
const qint32 width  = image.width();
const qint32 height = image.height();

// When we pass a frame to the encoder, it may keep a reference to it internally;
// make sure we do not overwrite it here!
av_frame_make_writable(tmpFrame);

// Converting QImage to AV_PIX_FMT_BGRA AVFrame ...
for (qint32 y = 0; y < height(); y++) {
    const uint8_t * scanline = image.scanLine(y);

    for (qint32 x = 0; x < width() * 4; x++) {
        tmpFrame->data[0][y * tmpFrame->linesize[0] + x] = scanline[x];
    }
}

// Make sure to clear the frame. It prevents a bug that displays only the
// first captured frame on the GIF export.
if (frame) {
    av_frame_free(&frame);
    frame = Q_NULLPTR;
}
frame = allocPicture(codecContext->width, codecContext->height, codecContext->pix_fmt);

if (yuvFrame) {
    av_frame_free(&yuvFrame);
    yuvFrame = Q_NULLPTR;
}
yuvFrame = allocPicture(codecContext->width, codecContext->height, AV_PIX_FMT_YUV420P);

// Converting BGRA -> YUV420P...
if (!swsCtx) {
    swsCtx = sws_getContext(width, height,
                            AV_PIX_FMT_BGRA,
                            width, height,
                            AV_PIX_FMT_YUV420P,
                            swsFlags, NULL, NULL, NULL);
}

// ...then converting YUV420P -> RGB8 (natif GIF format pixel)
if (!swsGIFCtx) {
    swsGIFCtx = sws_getContext(width, height,
                                AV_PIX_FMT_YUV420P,
                                codecContext->width, codecContext->height,
                                codecContext->pix_fmt,
                                this->swsFlags, NULL, NULL, NULL);
}

// This double scaling prevent some artifacts on the GIF and improve
// significantly the display quality
sws_scale(swsCtx,
          (const uint8_t * const *)tmpFrame->data,
          tmpFrame->linesize,
          0,
          codecContext->height,
          yuvFrame->data,
          yuvFrame->linesize);
sws_scale(swsGIFCtx,
          (const uint8_t * const *)yuvFrame->data,
          yuvFrame->linesize,
          0,
          codecContext->height,
          frame->data,
          frame->linesize);

...

AVPacket packet;
int gotPacket = 0;

av_init_packet(&packet);

// Packet data will be allocated by the encoder
packet.data = NULL;
packet.size = 0;

frame->pts = nextPts++; // nextPts starts at 0
avcodec_encode_video2(codecContext, &packet, frame, &gotPacket);

if (gotPacket) {
    // Rescale output packet timestamp values from codec to stream timebase
    av_packet_rescale_ts(packet, *codecContext->time_base, stream->time_base);
    packet->stream_index = stream->index;

    // Write the compressed frame to the media file.
    av_interleaved_write_frame(formatContext, packet);

    av_packet_unref(&this->packet);
}

Closing ffmpeg:

// Retrieving delayed frames if any...
// Note: mainly used for video generation, it might be useless for .gif.
for (int gotOutput = 1; gotOutput;) {
    avcodec_encode_video2(codecContext, &packet, NULL, &gotOutput);

    if (gotOutput) {
        // Rescale output packet timestamp values from codec to stream timebase
        av_packet_rescale_ts(packet, *codecContext->time_base, stream->time_base);
        packet->stream_index = stream->index;

        // Write the compressed frame to the media file.
        av_interleaved_write_frame(formatContext, packet);
        av_packet_unref(&packet);
    }
}

av_write_trailer(formatContext);

avcodec_free_context(&codecContext);
av_frame_free(&frame);
av_frame_free(&tmpFrame);
sws_freeContext(swsCtx);

if (!(outputFormat->flags & AVFMT_NOFILE)) {
    // Closing the output file...
    avio_closep(&formatContext->pb);
}

avformat_free_context(formatContext);

I don’t think it is the easiest way, but at least it worked for me. I let the question open. Please, feel free to comment/improve/answer this.

2
  • I began working with ffmpeg recently and your question with answer helped me a lot – massive thanks. Funny that the answer came a few hours after I found the question. This is faith! One thing I keep wondering about – you're using 3 frames and mentioning the issue with artefacts. Can you tell what sort of artefacts have you been getting? Does it look anything like this? Is there more info on this problem / solution? I'm getting some very-very-very strange results and really want to get to the bottom of this. I think it might be the issue you're describing… Commented Aug 26, 2016 at 19:52
  • I'm glad I could help you. You're right. If you're trying to convert bgra to rgb8 directly, using the code above, you will have this kind of artefacts. Unfortunately, I didn’t have time to expand my researches on that for now. It could be linked to an incompatibility between these two pixel formats.
    – Sierra
    Commented Aug 29, 2016 at 9:25
1

I made a more finished c++ class based on @Sierra's snippets. It does not use any deprecated ffmpeg functions so should be future-proof for now. The class is separated into three parts: A function to write the GIF image header into a GIF file, a function that inserts a (possibly cropped) image in a frame location, and a function to commit the frames into the file, flush it and remove allocated ffmpeg structures.

A couple things to note about this class:

  • This answer has a good explanation of why the maximum frame rate is 100FPS:

The GIF98a specification says in the section about the Graphics Control Extension

vii) Delay Time - If not 0, this field specifies the number of hundredths (1/100) of a second to wait before continuing with the processing of the Data Stream. The clock starts ticking immediately after the graphic is rendered. This field may be used in conjunction with the User Input Flag field.

Which means the smallest value is 1 for a image rate of 100 FPS (this will be difficult to render on most monitors...), and the next smallest value is 2 for an image rate of 50 FPS, and the next value is 3 for an image rate of 33.3 FPS. So exactly 60 FPS is not possible.

In other words, 60 FPS GIFs are actually saved at 100 FPS, 30 FPS at 50 FPS, and 15 FPS like what the OP is trying to use as 16.6 FPS.

  • I have noticed yellow artifacts in GIFs that have odd dimensions. Therefore, my class clips off the extra pixel of width/length.

  • AV_PIX_FMT_BGR8 frames worked for me so that is what my class creates.

  • Last and most importantly, frame numbers must be specified in order, beginning at 1. Trying to add frames out of order will make the class abort the encoding process until the next GIFExporter::commitFile() call. This was implemented because I was grabbing images from QOpenGLWidget frame buffers which lag if too many objects are drawn, causing some frames to be missed.

Github Gist code below:

<script src="https://gist.github.com/ZenulAbidin/4fb0047e459bd929a7c02784f78daca1.js"></script>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.