8

I have multiple images taken simultaneously pointing at the same direction from the same starting location. However, there is still a slight offset because these cameras were not in the exact same place when the picture was taking. I'm looking for a way to calculate the optimal translation/shear/skew/rotation needed to apply to match one image to another so that they overlay (almost) perfectly.

The images are in a .raw format which I am reading in 16 bits at a time.

I have been suggested (by my employer who is not a programmer [I'm an intern btw]) to take a portion of the source image (not at the edges) and brute-force search for a same-sized portion with a high correlation in data values. I'm hoping there is a less-wasteful algorithm.

2
  • If you really need to write this yourself, look up (for example) the SIFT algorithm. Unless you really need to, though, you're probably better off using something like Hugin or CombineZP that can already do the job. Jul 1, 2011 at 0:08
  • 1
    The correct term for this operation is image-registration. You will probably be able to find an answer from a search on SO.
    – rwong
    Jul 1, 2011 at 6:18

3 Answers 3

12

Here is a short code that does what you want (I use openCV 2.2):

  1. Suppose you have 2 images: srcImage,dstImage, and you want to align them
  2. The code is very simple. Use it as basis for your algorithm.

Code:

// Detect special points on each image that can be corresponded    
Ptr<FeatureDetector>  detector = new SurfFeatureDetector(2000);  // Detector for features

vector<KeyPoint> srcFeatures;   // Detected key points on first image
vector<KeyPoint> dstFeatures;
detector->detect(srcImage,srcFeatures);
detector->detect(dstImage,dstFeatures); 

// Extract descriptors of the features
SurfDescriptorExtractor extractor;  
Mat projDescriptors, camDescriptors;
extractor.compute(srcImage,  srcFeatures, srcDescriptors);
extractor.compute(dstImage , dstFeatures, dstDescriptors );

// Match descriptors of 2 images (find pairs of corresponding points)
BruteForceMatcher<L2<float>> matcher;       // Use FlannBasedMatcher matcher. It is better
vector<DMatch> matches;
matcher.match(srcDescriptors, dstDescriptors, matches);     

// Extract pairs of points
vector<int> pairOfsrcKP(matches.size()), pairOfdstKP(matches.size());
for( size_t i = 0; i < matches.size(); i++ ){
    pairOfsrcKP[i] = matches[i].queryIdx;
    pairOfdstKP[i] = matches[i].trainIdx;
}

vector<Point2f> sPoints; KeyPoint::convert(srcFeatures, sPoints,pairOfsrcKP);
vector<Point2f> dPoints; KeyPoint::convert(dstFeatures, dPoints,pairOfdstKP);

// Matched pairs of 2D points. Those pairs will be used to calculate homography
Mat src2Dfeatures;
Mat dst2Dfeatures;
Mat(sPoints).copyTo(src2Dfeatures);
Mat(dPoints).copyTo(dst2Dfeatures);

// Calculate homography
vector<uchar> outlierMask;
Mat H;
H = findHomography( src2Dfeatures, dst2Dfeatures, outlierMask, RANSAC, 3);

// Show the result (only for debug)
if (debug){
   Mat outimg;
   drawMatches(srcImage, srcFeatures,dstImage, dstFeatures, matches, outimg, Scalar::all(-1), Scalar::all(-1),
               reinterpret_cast<const vector<char>&> (outlierMask));
   imshow("Matches: Src image (left) to dst (right)", outimg);
   cvWaitKey(0);
}

// Now you have the resulting homography. I mean that:  H(srcImage) is alligned to dstImage. Apply H using the below code
Mat AlignedSrcImage;
warpPerspective(srcImage,AlignedSrcImage,H,dstImage.Size(),INTER_LINEAR,BORDER_CONSTANT);
Mat AlignedDstImageToSrc;
warpPerspective(dstImage,AlignedDstImageToSrc,H.inv(),srcImage.Size(),INTER_LINEAR,BORDER_CONSTANT);
8
  • Wow, thank you! I'm taking a second (more like 100th) look into the documentation and it's been staring me in the face! Thanks for highlighting where I needed to look at more carefully and the great example code.
    – andrw
    Jul 1, 2011 at 17:05
  • I can't seem to find the class FeatureDetector in my OpenCV installation (2.1), even though its in the documentation... any ideas? The first line: Ptr<FeatureDetector> detector = new SurfFeatureDetector(2000); gives me : 1>.\makeTforms.cpp(34) : error C2065: 'FeatureDetector' : undeclared identifier (VS 2008)
    – andrw
    Jul 1, 2011 at 18:20
  • I've reinstalled and I can't find that file anywhere. Did the installer change or something o_O
    – andrw
    Jul 1, 2011 at 22:49
  • I am using opnencv 2.2. In there the file is at: C:\Program Files\OpenCV2.2\modules\features2d\include\opencv2\features2d. Also in OpenCV2.2\bin I have the opencv_features2d220.dll and in lib library the opencv_features2d220.lib. Are you using windows? If so, try to installing from here: sourceforge.net/projects/opencvlibrary/files/opencv-win/2.2
    – DanielHsH
    Jul 2, 2011 at 10:37
  • Did you try to install from the link I posted?
    – DanielHsH
    Jul 3, 2011 at 15:00
0

Are the images taken standing from the same position but you're just rotated a bit so that they're not aligned correctly? If so then the images are related by a homography - i.e. a projective transformation. Given a set of correspondences between the images (you need at least 4 pairs), the standard way to find the homography is to use the DLT algorithm.

0

Avoid linker errors using the below code:

#include "cv.h"
#include "highgui.h"
using namespace cv;

// Directives to linker to include openCV lib files.
#pragma comment(lib, "opencv_core220.lib") 
#pragma comment(lib, "opencv_highgui220.lib") 
#pragma comment(lib, "opencv_contrib220.lib") 
#pragma comment(lib, "opencv_imgproc220.lib") 
#pragma comment(lib, "opencv_gpu220.lib") 
#pragma comment(lib, "opencv_video220.lib") 
#pragma comment(lib, "opencv_legacy220.lib") 

#pragma comment(lib, "opencv_ml220.lib") 
#pragma comment(lib, "opencv_objdetect220.lib") 
#pragma comment(lib, "opencv_ffmpeg220.lib") 

#pragma comment(lib, "opencv_flann220.lib") 
#pragma comment(lib, "opencv_features2d220.lib") 
#pragma comment(lib, "opencv_calib3d220.lib") 

// Your code here...
int main(void){
    Mat B = Mat:eye(3,3,CV_8U);
    return -1;
}

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.