I have been studying RetinaNet recently. I read the original paper and some related ones and wrote a post sharing what I have learnt: http://blog.zenggyu.com/en/post/2018-12-05/retinanet-explained-and-demystified/. However, I still have some confusions, which I also pointed out in the post. Can anyone please enlighten me?
Confusion #1
As indicated by the paper, an anchor box is assigned to background if its IoU with any ground-truth is below 0.4. In this case, what should be the corresponding classification target label (assuming there's K classes)?
I know that SSD has a background class (which makes K+1 classes in total), while YOLO predicts an confidence score indicating whether there is an object in the box (not background) or not (background) in addition to the K class probabilities. While I didn't find any statements in the paper indicating RetinaNet includes a background class, I did see this statement: "..., we only decode box predictions from ..., after thresholding detector confidence at 0.05", which seems to indicate that there's a prediction for confidence score. However, where does this score come from (since the classification subnet only outputs K numbers indicating the probability of K classes)?
If RetinaNet defines target labels differently from SSD or YOLO, I would assume that the target is a length-K vector with all 0s entries and no 1s. However, in this case how does the focal loss (see definition below) will punish an anchor if it is a false negative?
where
Confusion #2
Unlike many other detectors, RetinaNet uses a class-agnostic bounding box regressor, and the activation of the last layer of classification subnet is sigmoid activation. Does this mean that one anchor box can simultaneously predict multiple objects of different classes?
Confusion #3
Let's denote these matching pairs of anchor box and ground-truth box as ${(A^i, G^i)}_{i=1,...N}$, where $A$ represents an anchor, $G$ represents a ground-truth, and $N$ is the number of matches.
For each matching anchor, the regression subnet predicts four numbers, which we denote as $P^i = (P^i_x, P^i_y, P^i_w, P^i_h)$. The first two numbers specify the offset between the centers of anchor $A^i$ and ground-truth $G^i$, while the last two numbers specify the offset between the width/height of the anchor and the ground-truth. Correspondingly, for each of these predictions, there is a regression target $T^i$ computed as the offset between the anchor and the ground-truth:
Are the above equations correct?
Many thanks in advance and feel free to point out any other misunderstandings in the post!
Update:
For future reference, another confusion I had when I was studying RetinaNet (I found this conversation in slack):