17

I'm building an application that will take an image of a single person's whole body and will produce a "mugshot" for that person.

Mugshot meaning an image of the person's whole face, neck, hair and ears at the same general size of another mugshot.

Currently I'm using
http://askernest.com/archive/2008/05/03/face-detection-in-c.aspx
to implement OpenCV and I'm using

harrcascade_frontalface_default.xml  
harrcascade_frontalface_alt.xml  
harrcascade_frontalface_alt2.xml  
harrcascade_frontalface_alt_tree.xml

as my cascades.

I use all of the cascades because a single one will not detect all my faces. After I get all of the faces detected by all of the cascades I find my average square and use that for my final guess of how tall and wide the mugshot should be.

My problem is 3 parts.

  • My current process is rather slow. How can I speed up the detection process?
    Edit: I'm finding that the processing time is directly related to photo size. Reducing the size of the photos may prove to be helpful.

  • A single cascade will not detect all the faces I come across so I'm using all of them. This of course produces many varied squares and a few false positives. What method can I use to identify false positives and leave them out of the average square calculation? ex. SandmanWayne
    Edit : I'm implementing an average of values within standard deviation. Will post code soon.

  • I'm not exactly sure of the best way find the mugshot given the square coordinates of the face. Where can I find face to mugshot ratios?
    Edit : Solved this one. Assuming all my heads are ratios of their faces.

    static public Rectangle GetMugshotRectangle(Rectangle rFace)
    {
        int y2, x2, w2, h2;
    
        //adjust as neccessary
        double heightRatio = 2;
    
        y2 = Convert.ToInt32(rFace.Y - rFace.Height * (heightRatio - 1.0) / 2.0);
        h2 = Convert.ToInt32(rFace.Height * heightRatio);
        //height to width ratio is 1.25 : 1 in mugshots
        w2 = Convert.ToInt32(h2 * 4 / 5);
        x2 = Convert.ToInt32((rFace.X + rFace.Width / 2) - w2 / 2);
    
        return new Rectangle(x2, y2, w2, h2);
    }
    

    Sandman
    I just need to get rid of those false positives.

Ok make that 4 issues.

  • Our camera that we will be using is currently out of commission so I don't have a method of capturing images at the moment. Where can I find full body images of people that isn't pure pron like google's image search for full body images?
    Edit : "Person standing" Makes a good search :)
2
  • 2
    Is that Thomas Hayden Church?
    – Dan J
    Apr 21, 2011 at 20:43
  • It's the sandman in Spiderman haha Apr 21, 2011 at 21:01

4 Answers 4

1

A single cascade could do what all of your cascades do if it is set up this way, plus it does not give you several results to judge from. The cascades you use are maybe different in the collection of teaching pictures they are made of or in some parameters.

A tutorial on how to build an own cascade can be found here. It would be useful to get the pictures used to train the four cascades you use but I don't know if they are publicly available.

2
  • That is where I obtained my current cascades. I'm not sure I want to go through the effort of building my own cascade just yet. I don't exactly have 5000 images of 1000 unique faces. Apr 25, 2011 at 16:18
  • Maybe there are some "addition rules" which apply to cascades so a new cascade can be build from several ones that behave like if it where trained with all cascades images. But sounds complicated to me too..
    – dronus
    Apr 28, 2011 at 21:21
1

If you're reading this question in the 2020s, more recent solutions have been developed that simplify this task. Libraries that might be worth taking a look at include

To solve the mugshot task using FaceAiSharp you could use code similar to this:

  1. dotnet new console

  2. dotnet add package FaceAiSharp.Bundle
    dotnet add package Microsoft.ML.OnnxRuntime
    
  3. Edit Program.cs:

using FaceAiSharp;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;

// adjust these lines to load your own image
using var hc = new HttpClient();
var groupPhoto = await hc.GetByteArrayAsync(
    "https://upload.wikimedia.org/wikipedia/commons/1/1e/1942_JFK_uniform_portrait.jpg");
var img = Image.Load<Rgb24>(groupPhoto);

var det = FaceAiSharpBundleFactory.CreateFaceDetector();
FaceAiSharp.Applications.CropProfilePicture(det, img, scaleFactor: 1.5f);
img.Save("mugshot.jpg");
  1. dotnet run

Try different scaleFactor values to widen or narrow the crop.

Disclaimer: I'm the author of FaceAiSharp. It's MIT-licensed. Don't hesitate to open an issue if something doesn't work for you or could be improved.

1
  • 1
    Wow, library looks really good. I would have loved this 12.5 years ago hahaha. Sep 27, 2023 at 10:25
0

Ok, I figured it out but the project is on ice for the moment.
I do not have the source to paste as the VM takes forever to load up.
If someone is really interested, let me know and I'll post.
If you see something that looks like it could be done better let me know.

The steps I took were as follows.

  1. Load the image, if greater than 500 pixels in either height or width make a new image of a version scaled down to a maximum of 500px height or width. Save the scale.
  2. Run the C# OpenCV implementation at Ask Ernest on all of the frontal face Harrcasscades at Harrtraining.
  3. For some reason the rectangles produced by Ask Ernest need to be scaled up 1.3X
  4. With the generated rectangles use standard deviation to remove false positives. I did this for each corner of the rectangle finding the distance each rectangle corner was off from the average corner location. I made a note of any rectangle who had a corner out of whack and removed it from the list of candidate rectangles.
  5. Get the average rectangle from the remaining rectangles and apply "GetMugshotRectangle" from above in the question.
  6. Scale the mugshot rectangle back up using the scale we saved in step 1.
  7. Cut out the mugshot from the original image and save to it's new location.

Done!

0

I suggest you to use Upper-body Haar cascade file which will return you the rectangle till shoulder. Please find Head and shoulders cascade file at "http://alereimondo.no-ip.org/OpenCV/34"

Not the answer you're looking for? Browse other questions tagged or ask your own question.