3

I've been playing with the optical flow functions in OpenCV and am stuck. I've successfully generated X and Y optical flow fields/maps using the Farneback method, but I don't know how to apply this to the input image coordinates to warp the images. The resulting X and Y fields are of 32bit float type (0-1.0), but how does this translate to the coordinates of the input and output images? For example, 1.0 of what? The width of the image? The difference between the two?

Plus, I'm not sure what my loop would look like to apply the transform/warp. I've done plenty of loops to change color, but the pixels always remain in the same location. Moving pixels around is new territory for me!

Update: I got this to work, but the resulting image is messy:

//make a float copy of 8 bit grayscale source image
IplImage *src_img = cvCreateImage(img_sz, IPL_DEPTH_32F, 1);
cvConvertScale(input_img,src_img,1/255.0); //convert 8 bit to float

//create destination image
IplImage *dst_img = cvCreateImage(img_sz, IPL_DEPTH_32F, 1);

for(y = 0; y < flow->height; y++){

    //grab flow maps for X and Y
    float* vx = (float*)(velx->imageData + velx->widthStep*y);
    float* vy = (float*)(vely->imageData + vely->widthStep*y);

    //coords for source and dest image
    const float *srcpx = (const float*)(src_img->imageData+(src_img->widthStep*y));
    float *dstpx = (float*)(dst_img->imageData+(dst_img->widthStep*y));

    for(x=0; x < flow->width; x++)
    {
        int newx = x+(vx[x]);
        int newy = (int)(vy[x])*flow->width;
        dstpx[newx+newy] = srcpx[x];
    }
}

I could not get this to work. The output was just garbled noise:

cvRemap(src_img,dst_img,velx,vely,CV_INTER_CUBIC,cvScalarAll(0));
4
  • optical flow gives you speed of each block of pixels in X or Y direction Jun 16, 2011 at 9:08
  • Ok, so the float values returned from the flow maps are velocity values? Is that right? How is that velocity value applied to the pixel coordinates?
    – Synthetix
    Jun 16, 2011 at 9:17
  • Each block consists of pixels, each pixel of a block shares block's speed. Use sizes of an image and resulting speed matrix to calculate pixel-block mapping Jun 16, 2011 at 9:38
  • There are no "blocks", this is a dense optical flow algorithm, there is a velocity per pixel.
    – etarion
    Jun 16, 2011 at 9:56

2 Answers 2

5

The flow vectors are velocity values. If the pixel in image 1 at position (x, y) has the flow vector (vx, vy) it is estimated to be at position (x+vx, y+vy) (so the values aren't really in the [0, 1] range - they can be bigger, and be negative too). Easiest way to do the warping is to create floating point images with those values (x+vx for the x direction, similar for y), and then use cv::remap.

10
  • Ok, I took your advice and now I have it working on the X axis! I still need to figure out how to get the Y axis to update. And what is the purpose of using cvRemap()?
    – Synthetix
    Jun 16, 2011 at 11:07
  • it's exactly the same in x and y directions, there is no reason one should "work" and the other not. and cv::remap does the warping. You shouldn't use cvRemap if you are using c++, as your tags say, opencv has a distinct c++ interface.
    – etarion
    Jun 16, 2011 at 11:11
  • I updated the above post to show the code I am working with. I couldn't get cvRemap to work. I am using C syntax, the only reason I put C++ in the tags is because much of OpenCV is C++ based (I have to use g++ to compile).
    – Synthetix
    Jun 16, 2011 at 11:36
  • opencv works perfectly fine with C only - you can compile things with gcc. If you have to use g++ to compile, that's for some other reason.
    – etarion
    Jun 16, 2011 at 12:14
  • Sorry, let me clarify. The problem is I get a bunch of linker errors (Undefined symbols) if I set up Xcode to compile a C and not C++.
    – Synthetix
    Jun 16, 2011 at 12:36
2

Using OpenCV

https://github.com/opencv/opencv/blob/master/samples/python/opt_flow.py

def warp_flow(img, flow):
    h, w = flow.shape[:2]
    flow = -flow
    flow[:,:,0] += np.arange(w)
    flow[:,:,1] += np.arange(h)[:,np.newaxis]
    res = cv2.remap(img, flow, None, cv2.INTER_LINEAR)
    return res

Sample Result

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.