4

I am using GPUImage to process incoming video, and each frame I would like to determine the average luminosity of many rectangular subregions of the incoming image for hit detection purposes in a game, but I am having trouble doing so in a way that doesn't kill FPS.

My current situation is to loop through the hit boxes of interest, crop the frame of said hit boxes with GPUImageCropFilter, do average luminosity on the cropped region and in its completion block call hit method on main thread if luminosity is high enough. This works okay if there are just a few hit boxes, but for the project I am working on there could be dozens at a time which kills the FPS.

Are there any recommended ways to modify the above approach to improve performance? I am thinking it might be possible to accomplish this by making a new filter with a custom shader that does a kind of localized pixelation effect (pixellating the rectangles of interest, so then I can just check any pixel in that region for the luminosity), but I am not sure if it is possible to pass in an array of areas of interest like this to a filter / shader. Thanks.

2
  • I think you'll want to try downsampling instead of a pixellation filter. The pixellation filters throw away color information for all but one pixel, and there's no need to render something at that size. Downsampling can give a kind of crude averaging (particularly if using a Lanczos downsampling or something similar). The pixel data could then be read out using a raw data filter, but this approach will only work well if your areas to test are aligned in a grid.
    – Brad Larson
    Jun 12, 2013 at 20:23
  • Hmm; my hit areas aren't necessarily aligned in a grid. Basically what we are trying to accomplish is to capture the movements of micro-organisms on the camera and use them to "collect" moving objects in the game. At any given moment there could be multiple organisms being captured by the camera and dozens of these collectible game objects. It would be nice if there was a blob detection filter to give me center coordinates for perceived blobs, but I didn't see that. I saw GPUImageMotionDetector but that only returns one centroid, and I tried the Houghs Line detector, but that killed FPS. Jun 12, 2013 at 20:53

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy