3

I have a pretty good understanding of AlexNet and VGG. I could verify the number of parameters used in each layer with what is being submitted in their respective papers.

However when i try to do the same on the GoogleNet paper "Going Deeper With COnvolution", even after many iterations I am NOT able to verify the numbers they have in the 'Table 1' of their paper.

For example, the first layer is the good old plain convolution layer with kernel size (7x7), input number of maps 3 , output number of maps is 64. So based on this fact the number of parameters needed would be (3 * 49 * 64) + 64 (bias) which is around 9.5k but they say they use 2.7k. I did the math for other layers as well and i am always off by few percent than what they report. Any idea?

Thanks

5 Answers 5

6

I think the first line (2.7k) is wrong, but the rest of the lines of the table are correct.

Here is my computation: https://i.stack.imgur.com/4bDo9.jpg

my computation

Be care to check which input is connect to which layer, e.g. for the layer "inception_3a/5x5_reduce":

 input = "pool2/3x3_s2" with 192 channels
 dims_kernel = C*S*S =192x1x1
 num_kernel = 16 

Hence parameter size for that layer = 16*192*1*1 = 3072

1
  • Is the number of parameters 11,193,984 ? I wrote a simple script to calculate the number of parameters in caffe . here it is : pastebin.com/DqnQWxNY but I have heard Googlenet has 5M parameters! which one is correct? and here is the googlenet deploy : pastebin.com/dwftj0Cq
    – Hossein
    Mar 16, 2017 at 6:40
1

Looks like they divided the numbers by 1024^n to convert to the K/M labels on the number of parameters in the paper Table 1. That feels wrong. We're not talking about actual storage numbers here (as in "bytes"), but straight up number of parameters. They should have just divided by 1000^n instead.

1
  • Fully agree. Wasted half an hour scratching my head as I calculated 163696 parameters for inception 3a and was wondering where the 159K in the paper (table 1) came from.
    – Kai
    Nov 30, 2017 at 16:03
1

Maybe the 7 * 7 conv layer is actually the combination of 7 * 1 conv layer and 1 * 7 conv layer, then the num of params could be: ((7+7)643 + 64*2) / 1024 = 2.75k, which approaches 2.7k (or you can omit 128 biases).

As we know, Google introduced asymmetric convolution while doing spatial factorization in paper "Spatial Factorization into Asymmetric Convolutions"

0

(1x7+7x1)x3x64=2688≈2.7k, this is my opinion, I am a fresh student

1
  • 1
    provide more description of what you want to ask Jun 6, 2018 at 4:59
0

Number of parameters in a CONV layer would be : ((m * n * d)+1)* k), added 1 because of the bias term for each filter. The same expression can be written as follows: ((shape of width of the filter * shape of height of the filter * number of filters in the previous layer+1)*number of filters)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.