3

Is there a simple function in OpenCV to get the 3D position and pose of an object from a stereo camera pair?

I have the cameras and baseline calibrated with the chess board. I now want to take a known object like the same chessboard, with known 3D points in it's own coordinates and find the real world position (in the camera coordinates).

There are functions to do this for a single camera (POSIT) and functions to find the 3D disparity image for the entire scene.

It must be simple to do almost the same process as for the camera calibration and find the chessboard in the camera pair - but I can't find any function that takes object + image coords and returns camera coords for a stereo pair.

Thank you

2 Answers 2

1

After calibrating your stereo camera system, you have got the relative pose (=translation+orientation) between the two cameras. Using solvePnP/solvePnPRansac if you find the relative pose between one of the cameras and the object and then consequently you have got the relative pose between the object and the other camera as well. For example, in stereo systems used for robot navigation usually reconstructed 3D points from previous frames are matched against only one of the cameras and then the relative camera pose from the 3d points is estimated. The stereo system just eases and improves the quality of triangulation/structure reconstruction.

8
  • Yes there are functions to do pose from a single camera, and to do stereo from a dense image pair - but there is nothing to do the 3D position from corresponding points in a stereo pair? This is just what the calibrate chessboard is doing internally - but I wanted some simple function to call. Aug 17, 2012 at 17:24
  • You seem to be confused. While calibrating you don't have the relative pose between the two cameras, hence the procedure is involved. After calibration you have the relative pose, all you need is the relative between one of the cameras and the object. Then you have the relative pose of the object to both cameras. What else do you need to get?
    – fireant
    Aug 17, 2012 at 17:31
  • Sorry wasn't clear. I have calibrated a camera pair with a chessboard - I now want to take a similar chessboard and attach it to an object and have the position+rotation of the object from the camera pair. Aug 17, 2012 at 18:01
  • 1
    So if you have measured the distance between the corners of the grid on the board then as I said above you are dealing with a single camera pose estimation. A stereo cam system is nothing more than a camera whose coordinate system is considered the world coordinate system and a second camera whose pose in that coordinate system is known. Try to think about it and make it clear to yourself. If something doesn't make sense then ask a more specific question.
    – fireant
    Aug 17, 2012 at 18:08
  • Yes the pose can be estimated from a single camera if you know the shape. But with a stereo pair I also have a direct 'Z' measurement from the disparity. I thought this could be used to make a more accurate pose if I had an x,y,z for every corner. Aug 17, 2012 at 18:17
0

Yes, StereoBM_create():

import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt

imgL = cv.imread('tsukuba_l.png',0)
imgR = cv.imread('tsukuba_r.png',0)

stereo = cv.StereoBM_create(numDisparities=16, blockSize=15)
disparity = stereo.compute(imgL,imgR)

plt.imshow(disparity,'gray')
plt.show()

https://docs.opencv.org/4.x/dd/d53/tutorial_py_depthmap.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.