3

I am trying to convert an RGB frame, which is taken from OpenGL glReadPixels(), to a YUV frame, and write the YUV frame to a file (.yuv). Later on I would like to write it to a named_pipe as an input for FFMPEG, but as for now I just want to write it to a file and view the image result using a YUV Image Viewer. So just disregard the "writing to pipe" for now.

After running my code, I encountered the following errors:

  1. The number of frames shown in the YUV Image Viewer software is always 1/3 of the number of frames I declared in my program. When I declare fps as 10, I could only view 3 frames. When I declared fps as 30, I could only view 10 frames. However when I view the file in Text Editor, I could see that I have the correct amount of word "FRAME" printed in the file. This is the example output that I got: http://www.bobdanani.net/image.yuv

  2. I could not see the correct image, but just some distorted green, blue, yellow, and black pixels.

I read about YUV format from http://wiki.multimedia.cx/index.php?title=YUV4MPEG2 and http://www.fourcc.org/fccyvrgb.php#mikes_answer and http://kylecordes.com/2007/pipe-ffmpeg

Here is what I have tried so far. I know that this conversion approach is quite in-efficient, and I can optimize it later. Now I just want to get this naive approach to work and have the image shown properly.

int frameCounter = 1; 
int windowWidth = 0, windowHeight = 0;
unsigned char *yuvBuffer;
unsigned long bufferLength = 0;
unsigned long frameLength = 0;
int fps = 10;

void display(void) {

    /* clear the color buffers */
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    /* DRAW some OPENGL animation, i.e. cube, sphere, etc 
     .......
     .......
    */

    glutSwapBuffers();

    if ((frameCounter % fps) == 1){
        bufferLength = 0;
        windowWidth = glutGet(GLUT_WINDOW_WIDTH);
        windowHeight = glutGet (GLUT_WINDOW_HEIGHT);
        frameLength = (long) (windowWidth * windowHeight * 1.5 * fps) + 100; // YUV 420 length (width*height*1.5) + header length
        yuvBuffer = new unsigned char[frameLength];
        write_yuv_frame_header();
    }

    write_yuv_frame();

    frameCounter = (frameCounter % fps) + 1;

    if ( (frameCounter % fps) == 1){
        snprintf(filename, 100, "out/image-%d.yuv", seq_num);
        ofstream out(filename, ios::out | ios::binary); 
        if(!out) { 
            cout << "Cannot open file.\n"; 
        } 

        out.write (reinterpret_cast<char*> (yuvBuffer), bufferLength);
        out.close();
        bufferLength = 0;
        delete[] yuvBuffer;
    }
}


void write_yuv_frame_header (){
    char *yuvHeader = new char[100];
    sprintf (yuvHeader, "YUV4MPEG2 W%d H%d F%d:1 Ip A0:0 C420mpeg2 XYSCSS=420MPEG2\n", windowWidth, windowHeight, fps);
    memcpy ((char*)yuvBuffer + bufferLength, yuvHeader, strlen(yuvHeader));
    bufferLength += strlen (yuvHeader);
    delete (yuvHeader);
}

void write_yuv_frame() {
    int width = glutGet(GLUT_WINDOW_WIDTH);
    int height = glutGet(GLUT_WINDOW_HEIGHT);
    memcpy ((void*) (yuvBuffer+bufferLength), (void*) "FRAME\n", 6);
    bufferLength +=6;

    long length = windowWidth * windowHeight;
    long yuv420FrameLength = (float)length * 1.5;
    long lengthRGB = length * 3;
    unsigned char *rgb      = (unsigned char *) malloc(lengthRGB * sizeof(unsigned char));
    unsigned char *yuvdest  = (unsigned char *) malloc(yuv420FrameLength * sizeof(unsigned char));
    glReadPixels(0, 0, windowWidth, windowHeight, GL_RGB, GL_UNSIGNED_BYTE, rgb);

    int r, g, b, y, u, v, ypos, upos, vpos;

    for (int j = 0; j <  windowHeight; ++j){
        for (int i = 0; i < windowWidth; ++i){
            r = (int)rgb[(j * windowWidth + i) * 3 + 0];
            g = (int)rgb[(j * windowWidth + i) * 3 + 1];
            b = (int)rgb[(j * windowWidth + i) * 3 + 2];

            y = (int)(r *  0.257 + g *  0.504 + b *  0.098) + 16;
            u = (int)(r *  0.439 + g * -0.368 + b *  -0.071) + 128;
            v = (int)(r *  -0.148 + g * -0.291 + b * 0.439 + 128);

            ypos = j * windowWidth + i;
            upos = (j/2) * (windowWidth/2) + i/2 + length;
            vpos = (j/2) * (windowWidth/2) + i/2 + length + length/4;

            yuvdest[ypos] = y;
            yuvdest[upos] = u;
            yuvdest[vpos] = v;            
        } 
    }

    memcpy ((void*) (yuvBuffer + bufferLength), (void*)yuvdest, yuv420FrameLength);
    bufferLength += yuv420FrameLength;
    free (yuvdest);   
    free (rgb);
}

This is just the very basic approach, and I can optimize the conversion algorithm later. Can anyone tell me what is wrong in my approach? My guess is that one of the issues is with the outstream.write() call, because I converted the unsigned char* data to char* data that it may lose data precision. But if I don't cast it to char* I will get a compile error. However this doesn't explain why the output frames are corrupted (only account to 1/3 of the number of total frames).

1
  • Have you solved this ?
    – Michael IV
    Mar 12, 2013 at 16:10

2 Answers 2

1

It looks to me like you have too many bytes per frame for 4:2:0 data. ACcording to the spec you linked to, the number of bytes for a 200x200 pixel 4:2:0 frame should be 200 * 200 * 3 / 2 = 60,000. But you have ~90,000 bytes. Looking at your code, I don't see where you are convert from 4:4:4 to 4:2:0. So you have 2 choices - either set the header to 4:4:4, or convert the YCbCr data to 4:2:0 before writing it out.

4
  • I already set the number of bytes to 200 * 200 * 3 / 2 (in the yuv420FrameLength). So the size of yuv buffer is 60,000 for 200*200 pixel image. I also set the frame length to width * height * 1.5 * fps. Furthermore, the following codes position the data in 4:2:0 format: ypos = j * windowWidth + i; upos = (j/2) * (windowWidth/2) + i/2 + length; vpos = (j/2) * (windowWidth/2) + i/2 + length + length/4; yuvdest[ypos] = y; yuvdest[upos] = u; yuvdest[vpos] = v; Mar 25, 2012 at 15:16
  • Ah, sorry. I downloaded your output file and when I looked at it in a text editor, there were about 90,000 bytes between the first "FRAME" and second "FRAME", but it turns out that's because it got converted to UTF8 by the text editor. Looking at it in a hex editor, there are only 60,000 bytes, which seems correct. Mar 25, 2012 at 16:43
  • Can you post the before and after pictures? That might make it more obvious what's going on. Also, what happens if you write out 4:4:4 data (with the proper header)? Do you get all frames then? One other thing - I notice the wikipedia link mentions that the headers starts with 'YUVMPEG4 ' (with a space at the end), and that all fields start with a space. Do you need 2 spaces between the YUV marker and the width marker? (It's not clear to me from wikipedia whether that's the case.) Mar 25, 2012 at 16:48
  • Can you post the code where you write out the data? Also, can you verify in the debugger that when you convert the u and v from signed ints to unsigned chars that you get the correct bit pattern? Mar 25, 2012 at 17:07
0

I compiled your code and surely there is a problem when computing upos and vpos values. For me this worked (RGB to YUV NV12):

vpos = length + (windowWidth * (j/2)) + (i/2)*2;
upos = vpos + 1;
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.