1

I have been looking into opencv and want to match images taken from 2 different cameras. Basically a Kinect and an external camera. I found homography to be a good technique but have loads of questions. After undistorting the images, do I have to find matching points again (by using the chessboard) on the undistorted images to get the Homography matrix? Does the chessboard need to have perfect squares? Is there any simple solution for this problem?

Thanks!

3 Answers 3

2

The cameras are your measurement devices. You've calibrated them to get rid of inaccuracies, and to get the correct homography, you need to use this undistorted, accurate data. To compute the homography, you now need to find features in the 1st image and their matches in the 2nd image. The points found in both images (or at least some subset of them) must correspond to world points that lie on a plane. The set of matches does not need to be perfect, as the homography computaton procedure in OpenCV can use RANSAC for robust estimation and as such is capable of dealing with outliers (at least to some degree).

From what I understand, in your application you can have a calibration pattern visible for both of cameras, which makes things even easier - just look for the chessboards in both images and establish matches, then pass it to the homograpy computation function. You can be sure the world points all lie on a plane. As for the square shape and size - it does matter for calibration, not for homography computation as long as both cameras see the same pattern, but you may have some trouble finding a non-accurate chessboard using OpenCV functions.

2

Thanks guys. I ended up finding the Homography matrix between two distorted(normal) images, and making a perspective transform. It was clearly the simplest way and it worked, didn't seem to get much better results from a calibrated image :D

0

Oh, I had the same problem and I solved it in a different way. I simply used a small calibration: ask the user to position his/her hand on a specific point on the camera image, then captured that position. Repeat 3 more times to get the 4 corners of a rectangle. Then, I used a formula to map a point from a trapezoid to a rectangle, and voila! Now I have converted coordinates from the Kinect to the camera space.

It has limitations: the Kinect and the camera must be as close as possible, or else when the user leaves the plane he's standing in (moves closer or further to the Kinect) inaccuracies appear. But it works well enough for most cases, test it and you'll see.

I even made a small library (in C# and in AS3) for that, send me a message after 6 pm (GMT -5) and I'll send it to you or post it here, with a small demo application; and also edit and post the link to the site where I found the formula, which I can't find again now.

My team also struggled with OpenCV's homography for two weeks without results, until we came up with this.

2
  • If I have the pose between the cameras and the camera matrix, can i solve for the homography matrix?
    – raaj
    Aug 26, 2016 at 7:52
  • If you have two Kinect cameras, and have 3D points from the two of them, I guess you can. But the OP referred to an external camera, which does not provide 3D information. Aug 26, 2016 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.