0

I just saw a paper which says the empirical minimal number is \sqrt_{n_{in}+n_{out}}+1. Andrew said every hidden layer can have the same number of neurons. So, is there any good idea about this?

1
  • There is not a "best" answer to this question and it is highly problem specific. Jeff Hinton says twice more neurons into the first hidden layer than the input layer gives "most of the time" goods results. But years ago, it was recommended to use a pyramidal shape: input > hidden > output.
    – FiReTiTi
    Oct 13, 2015 at 8:47

1 Answer 1

0

A common solution to this problem is to grid search multiple configurations of the neural network hidden layer, and use a validation set to compare error rates. This should give you a reasonable estimate of the most likely network configuration to generalize to new data best.

In general having too few hidden neurons will not allow your neural network to learn the patterns within your data set. Having too many hidden neurons will allow your network to overfit the patterns in the data set as it has enough available "memory" to learn the spurious patterns of noise.

1
  • Yeah, I know. The problem is the training time of NN is often quite long like several days, so grid search will take too much time cuz there can be many choices.
    – Crazymage
    Oct 15, 2015 at 9:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.