18

I use this to functions as a base of my tracking algorithm.

    //1. detect the features
    cv::goodFeaturesToTrack(gray_prev, // the image 
    features,   // the output detected features
    max_count,  // the maximum number of features 
    qlevel,     // quality level
    minDist);   // min distance between two features

    // 2. track features
    cv::calcOpticalFlowPyrLK(
    gray_prev, gray, // 2 consecutive images
    points_prev, // input point positions in first im
    points_cur, // output point positions in the 2nd
    status,    // tracking success
    err);      // tracking error

cv::calcOpticalFlowPyrLK takes vector of points from the previous image as input, and returns appropriate points on the next image. Suppose I have random pixel (x, y) on the previous image, how can I calculate position of this pixel on the next image using OpenCV optical flow function?

2 Answers 2

29

As you write, cv::goodFeaturesToTrack takes an image as input and produces a vector of points which it deems "good to track". These are chosen based on their ability to stand out from their surroundings, and are based on Harris corners in the image. A tracker would normally be initialised by passing the first image to goodFeaturesToTrack and obtaining a set of features to track. These features could then be passed to cv::calcOpticalFlowPyrLK as the previous points, along with the next image in the sequence and it will produce the next points as output, which then become input points in the next iteration.

If you want to try to track a different set of pixels (rather than features generated by cv::goodFeaturesToTrack or a similar function), then simply provide these to cv::calcOpticalFlowPyrLK along with the next image.

Very simply, in code:

// Obtain first image and set up two feature vectors
cv::Mat image_prev, image_next;
std::vector<cv::Point> features_prev, features_next;

image_next = getImage();

// Obtain initial set of features
cv::goodFeaturesToTrack(image_next, // the image 
  features_next,   // the output detected features
  max_count,  // the maximum number of features 
  qlevel,     // quality level
  minDist     // min distance between two features
);

// Tracker is initialised and initial features are stored in features_next
// Now iterate through rest of images
for(;;)
{
    image_prev = image_next.clone();
    feature_prev = features_next;
    image_next = getImage();  // Get next image

    // Find position of feature in new image
    cv::calcOpticalFlowPyrLK(
      image_prev, image_next, // 2 consecutive images
      points_prev, // input point positions in first im
      points_next, // output point positions in the 2nd
      status,    // tracking success
      err      // tracking error
    );

    if ( stopTracking() ) break;
}
5
  • 1
    I notice you only do feature detection for one time. I have tested this code. I found only the features detected on the first image can be tracked. If all these features go beyond of the image, there would be no feature to track. I need to use optical flow for 3D construction. Then how can we continuously tracking old features and in the meantime add new image features? Thanks.
    – Shiyu
    Commented Apr 15, 2012 at 2:53
  • 1
    Yes, you only detect features with goodFeaturesToTrack, then the optical flow method simply tracks them. If you want to maintain a certain number of features in each frame, you would have to detect how many features were successfully tracked to the current frame and then attempt to detect additional ones to be tracked to the next frame. An alternative would be to detect features in every frame, and then calculate descriptors and match those descriptors by using functions on this page.
    – Chris
    Commented Apr 16, 2012 at 8:37
  • If you need more detail, it would be better to ask a new question.
    – Chris
    Commented Apr 16, 2012 at 8:37
  • Since I work with video sequences, I prefer to use optical flow for feature matching. If I detect features in every frame, the features would not be tracked stably because the features detected this time may not be detected next time. Thanks for the reply. But I don't understand how to "detect additional ones"? For example, if I detect 100 features in the first frame, now only 50 in the field of view, how can I detect additional 50 features while maintaining the old 50? I think the only solution is to run goodFeaturesToTrack to detect a new set of 100 features, right?
    – Shiyu
    Commented Apr 16, 2012 at 12:55
  • I have posted a new question about this: stackoverflow.com/questions/10159236/…
    – Shiyu
    Commented Apr 16, 2012 at 12:56
1

cv::calcOpticalFlowPyrLK(..) function uses arguments :

cv::calcOpticalFlowPyrLK(prev_gray, curr_gray, features_prev, features_next, status, err);

cv::Mat prev_gray, curr_gray;
std::vector<cv::Point2f> features_prev, features_next;
std::vector<uchar> status;
std::vector<float> err;

simplest(partial) code to find pixel in next frame :

features_prev.push_back(cv::Point(4, 5));
cv::calcOpticalFlowPyrLK(prev_gray, curr_gray, features_prev, features_next, status, err);

If pixel was successfully found status[0] == 1 and features_next[0] will show coordinates of pixel in next frame. Value information can be found in this example: OpenCV/samples/cpp/lkdemo.cpp

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.