please i like to classify a set of image in 4 class with SIFT DESCRIPTOR and SVM. Now, using SIFT extractor I get keypoints of different sizes exemple img1 have 100 keypoints img2 have 55 keypoints.... how build histograms that give fixed size vectors with matlab
3 Answers
In this case, perhaps dense sift is a good choice.
There are two main stages:
Stage 1: Creating a codebook.
- Divide the input image into a set of sub-images.
- Apply sift on each sub-image. Each key point will have 128 dimensional feature vector.
- Encode these vectors to create a codebook by simply applying k-means clustering with a chosen
k
. Each image will produce a matrix Vi (i <= n
andn
is the number of images used to create the codeword.) of size128 * m
, wherem
is the number of key points gathered from the image. The input to K-means is therefore, a big matrix V created by horizontal concatenation of Vi, for alli
. The output of K-means is a matrix C with size128 * k
.
Stage 2: Calculating Histograms.
For each image in the dataset, do the following:
- Create a histogram vector
h
of sizek
and initialize it to zeros. - Apply dense sift as in step 2 in stage 1.
- For each key point's vector find the index of it's "best match" vector in the codebook matrix C (can be the minimum in the Euclidian distance) .
- Increase the corresponding bin to this index in
h
by 1. - Normalize
h
by L1 or L2 norms.
Now h
is ready for classification.
Another possibility is to use Fisher's vector instead of a codebook, https://hal.inria.fr/file/index/docid/633013/filename/jegou_aggregate.pdf
-
Could you explain how testing and training processes should go? I'm trying to implement the approach you've described here and can't understand how to test the model. If I have a labelled dataset of images, firstly, I create a codebook for the test image, right? Then, for each image in the dataset, I calculate histograms using the codebook of the test image. So, I get a bunch of histograms on which I can train my SVM. But how do I test it? How do I get my single input image meant for testing to the format of those histograms, so I can test my trained SVM on it? Or my understanding is wrong? Jan 18, 2017 at 19:13
-
1Ok, no I think I get this. I should create a codebook not for the "input image", but for the whole dataset. And only then create histograms suing the generated codebook. Jan 18, 2017 at 19:47
You will always get different number of keypoints for different images, but the size of feature vector of each descriptor point remains same i.e. 128. People prefer using Vector Quantization or K-Mean Clustering and build Bag-of-Words model histogram. You can have a look at this thread.
Using the conventional SIFT approach you will never have the same number of key points in every image. One way of achieving that is to sample the descriptors densely, using Dense SIFT, that places a regular grid on top of the image. If all images have the same size, then you will have the same number of key points per image.