8

I am attempting to use OpenCV to grab frames from a webcam and display them in a window using SFML.

VideoCapture returns frames in OpenCV's Mat format. To display the frames, SFML requires a 1D array of pixels in its uint8 format, which (as far as I can tell) is interchangeable with uchar. This array is expected to represent 32 bits per pixel RGBA.

So, I have a uchar array, and I'm looping over the Mat data and copying each pixel:

VideoCapture cap(0);
Mat frame;
cap >> frame;

uchar* camData = new uchar[640*480*4];
uchar* pixelPtr = frame.data;
for(int i = 0; i < frame.rows; i++)
{
    for(int j = 0; j < frame.cols; j++)
    {
        camData[i*frame.cols + j + 2] = pixelPtr[i*frame.cols + j + 0]; // B
        camData[i*frame.cols + j + 1] = pixelPtr[i*frame.cols + j + 1]; // G
        camData[i*frame.cols + j + 0] = pixelPtr[i*frame.cols + j + 2]; // R
        camData[i*frame.cols + j + 3] = 255;

    }
}
img.LoadFromPixels(640, 480, camData); //Load pixels into SFML Image object for display

Unfortunately, this doesn't quite work. Something in that loop is wrong, as the resulting image when I load and display camData is scrambled.

As far as I can discern, either my math in the loop is wrong so the pixels are being assigned wrong, or the Mat data is in some format other than BGR.

Any ideas?

3
  • What exactly do you mean by scrambled? Could you maybe post an example of the resulting image?
    – sietschie
    Apr 22, 2012 at 6:32
  • 1
    Example. The pixel data from the source image isn't ending up in the correct locations in the destination array, so it makes weird interleaving patterns.
    – Thew
    Apr 22, 2012 at 7:26
  • just had the same pattern after pulling data back from the gpu/opencl and dumping it to file :D
    – Mark Essel
    Oct 24, 2013 at 15:17

3 Answers 3

12

OpenCV can do all job for you:

VideoCapture cap(0);
Mat frame;
cap >> frame;

uchar* camData = new uchar[frame.total()*4];
Mat continuousRGBA(frame.size(), CV_8UC4, camData);
cv::cvtColor(frame, continuousRGBA, CV_BGR2RGBA, 4);
img.LoadFromPixels(frame.cols, frame.rows, camData);
1
  • This LoadFromPixels method does not seem to be here anymore in OpenCV 4, is there a replacement ? May 13, 2022 at 12:29
3

I like the accepted answer better but this snippet helps you understand what's going on.

 for (int i=0; i<srcMat.rows; i++) {
            for (int j=0; j<srcMat.cols; j++) {
                int index = (i*srcMat.cols+j)*4;
                // copy while converting to RGBA order
                dstRBBA[index + 0] = srcMat[index + 2 ];
                dstRBBA[index + 1] = srcMat[index + 1 ];
                dstRBBA[index + 2] = srcMat[index + 0 ];
                dstRBBA[index + 3] = srcMat[index + 3 ];
            }
        }
1

For me worked following code:

VideoCapture capture(0);
Mat mat_frame;
capture >> mat_frame; // get a new frame from camera            

// Be sure that we are dealing with RGB colorspace...
Mat rgbFrame(width, height, CV_8UC3);
cvtColor(mat_frame, rgbFrame, CV_BGR2RGB);

// ...now let it convert it to RGBA
Mat newSrc = Mat(rgbFrame.rows, rgbFrame.cols, CV_8UC4);
int from_to[] = { 0,0, 1,1, 2,2, 3,3 };
mixChannels(&rgbFrame, 2, &newSrc, 1, from_to, 4);

The result (newSrc) is a premultiply image!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.