10

I am using Azure cognitive-service custom vision service to detect shapes from capture images. As per their documentation, I got the response as per their format.

But I am an facing issue to plot the shape above the image.

{
    "id": "0fbda4ee-8956-4979-bf57-a252441af98d",
    "project": "9ca4032b-beeb-40ad-9396-1c3fcfd9ba89",
    "iteration": "27c85265-a158-4fc4-b22a-d535dd758d80",
    "created": "2018-06-11T09:34:29.9496528Z",
    "predictions": [
        {
            "probability": 0.0102891214,
            "tagId": "677afcf8-bc4a-493f-b588-707663286125",
            "tagName": "ball",
            "boundingBox": {
                "left": 0.2889924,
                "top": 0.0169312358,
                "width": 0.7007024,
                "height": 0.8284572
            }
        },
        {
            "probability": 0.012788726,
            "tagId": "ca844f08-b6c0-4d9a-9010-73945d442708",
            "tagName": "cricket ball",
            "boundingBox": {
                "left": 0.304018974,
                "top": 0.413163722,
                "width": 0.299461246,
                "height": 0.436399817
            }
        },
        {
            "probability": 0.0229086485,
            "tagId": "ca844f08-b6c0-4d9a-9010-73945d442708",
            "tagName": "cricket ball",
            "boundingBox": {
                "left": 0.2889924,
                "top": 0.0169312358,
                "width": 0.7007024,
                "height": 0.8284572
            }
        },
        {
            "probability": 0.0100123268,
            "tagId": "4672144d-5593-446f-be63-5144a35d0e6e",
            "tagName": "pipe",
            "boundingBox": {
                "left": 0.711509764,
                "top": 0.377838552,
                "width": 0.07217276,
                "height": 0.113578767
            }
        },
        {
            "probability": 0.0167990718,
            "tagId": "4672144d-5593-446f-be63-5144a35d0e6e",
            "tagName": "pipe",
            "boundingBox": {
                "left": 0.9821227,
                "top": 0.9500536,
                "width": 0.0115685463,
                "height": 0.033854425
            }
        },
        {
            "probability": 0.923659563,
            "tagId": "4672144d-5593-446f-be63-5144a35d0e6e",
            "tagName": "pipe",
            "boundingBox": {
                "left": 0.288039029,
                "top": 0.411838,
                "width": 0.291451037,
                "height": 0.4237842
            }
        }
    ]
}

Above is the response I got in that Custom vision API call. But the issue is with boundingBox. It is having values always in a fraction, starting from 0. Now if I want to use that and want to draw a square, then it is not possible as because I don't know exact logic behind drawing square/rectangles from that values.

How can I use those values and draw a rectangles/square using it?

1 Answer 1

21

Reply / TL;DR

These boundingBox values are in percent of the image original size, so you can draw the rectangle by multiplying the values by the image width (for left and width values) or by the image height (for top and height values).

Keep in mind that the position is expressed from the top left corner, so position 0,0 is this corner.

Details with a sample

I got a small custom vision detecting cola bottles.

Original image is the following one: original image

I used the Custom Vision portal to make a prediction and got the following result - let's focus on this highlighted result with a 87,5% score:

portal result

Using the API (available here), I also made the Predict operation and got (among other details) this prediction:

{
    "probability": 0.875464261,
    "tagId": "1932c95f-ed4a-4675-bde4-c2457e1389e6",
    "tagName": "CocaLight",
    "boundingBox": {
      "left": 0.453497916,
      "top": 0.0,
      "width": 0.2523211,
      "height": 0.8738168
    }
}

Considering that my image dimension is 800 x 652 (so ImageWidth 800, ImageHeight 652):

Rectangle draw

Top left point position?

  • x (vertical distance from the left border) = left value from API x ImageWidth => 0.453497916 x 800 = 362
  • y (horizontal distance from the top border) = top value from the API x ImageHeight => 0.0 x 652 = 0

So my rectangle starting position is (362,0).

Size?

  • Rectangle width = width from the API x ImageWidth => 201
  • Rectangle height = height from the API x ImageHeight => 569

Let's draw it!

Draw API Result

Looks right!

2
  • 2
    Nice well explained.easy to understand.I'll try it and let you know. Thank you. Jun 12, 2018 at 4:36
  • 2
    @Nicolas R Really nice explanation. I hope cognitive service will update their description like this for drawing shapes. Really nice. Thanks a lot. I will try it out. Jun 12, 2018 at 4:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.