In an application I've to analyse movie files (let's say compute the differences of consecutive frame pairs). For this I use opencv (which uses ffmpeg as lib/ codecs). Depending on the video format there are different cpu loads/ uses. For wmv3 there seems to be not more than 1 core used. So it was close by hand to let multiple threads work on different parts of the movie, as the data is independent (beside having to stitch the parts afterwards). The code (stripped by the lap-parameter) is quite simple:
int main(int argc, char *argv[])
{
const string source = "move.wmv";
VideoCapture capt(source);
if (!capt.isOpened())
{
cout << "Could not open file " << source << endl;
return -1;
}
unsigned short nThreads (8);
double *pDiffArray = new double [(size_t) (capt.get(CV_CAP_PROP_FRAME_COUNT)];
capt.release();
ComputeDifferences (source, pDiffArray, nThreads);
return 0;
}
int ComputeDifferences (const string& source, double *pDiffArray, const unsigned short& nThreads)
{
std::vector<std::thread *> threadVector;
for (unsigned int i=0; i< nThreads; i++)
threadVector.push_back (new std::thread (ComputePart, source, pDiffArray, nThreads, i));
for (unsigned int i=0; i< nThreads; i++)
threadVector.at (i)->join();
// Stitching
;
return 0;
};
void ComputePart (const string source, double *pDiffArray,
const unsigned int& nThreads, const unsigned int& nThreadNo)
{
VideoCapture capt(source);
if (!capt.isOpened())
{
cout << "Could not open file " << source << endl;
}
size_t startPosDiffArray;
startPosDiffArray = nThreadNo * (capt.get(CV_CAP_PROP_FRAME_COUNT) / nThreads);
size_t sizePart (capt.get(CV_CAP_PROP_FRAME_COUNT) / nThreads);
size_t startPosFrame;
startPosFrame = capt.get(CV_CAP_PROP_FRAME_COUNT) / nThreads * nThreadNo;
capt.set(CAP_PROP_POS_FRAMES, startPosFrame);
Size refS = Size((int) capt.get(CAP_PROP_FRAME_WIDTH),
(int) capt.get(CAP_PROP_FRAME_HEIGHT));
Mat frame, frameRes;
std::array<Mat, 2> frameDuo;
Scalar s;
capt >> frameDuo [0];
if (!frameDuo [0].data)
return;
for (size_t i = 1; i < sizePart; i++) {
capt >> frameDuo [i%2];
if (!frameDuo [i%2].data)
break;
absdiff (frameDuo [(i-1)%2], frameDuo [i%2], frameRes);
s = sum (frameRes);
pDiffArray [i-1+startPosDiffArray] = (s [0] + s [1] + s [2])/ (refS.height * refS.width);
}
capt.release();
}
If I use it on a wmv3 video, 1280x720, abt. 50,000 frames, I get this speedups (at an Intel i7), relative to single thread (190 sec).
- MT2 1.8
- MT4 2.6
- MT8 3.0
Beside being very disappointed I do not understand what is happening here. I do know Amdahl's law etc., but in this case I expected a far better speedup. Does anyone have a hint for me (being a newbie on that)? It's not the positioning (capt.set ()), as disabling that doesn't change anything. Is it about ffmpeg-lib, opencv, thread-switch of std-lib, working set problem?
[Edit:
As of a hint in the comments I found that 80% of the time is used in
capt >> frameDuo [i%2];
This consists of reading from file, decoding and copying into opencv structure. And from this only the reading from file is of "sequential type" (in Amdahl's sense). As the HDD doesn't show heavy access (even when MT8), and there is no difference when using a quick SSD I don't understand why this sequential part should have such a big effect. How is it possible that 8 cores are fully working but only have a speedup of 3? And: how can I do better?]
VideoCapture capt(source);
doing?