Final Edit: Cleaned up the question and accepted runDOSrun's answer. IVlad's is equally good, and user3760780's is extremely helpful too. I recommend reading all three of those as well as the comments. The TLDR answer is that Possibility #1 is more or less the correct one but I phrased it very badly.
What does the input layer consist of in Neural Networks? What does that layer do?
A similar question is here Neural Networks: Does the input layer consist of neurons? but the answers there did not clear up my confusion.
Like the poster in the question above, I'm confused by the many contradicting things the Internet has to say about the input layer of a basic feed-forward network.
I'll skip the links to contradicting tutorials and articles and list the three possibilities that I can see. Which one (if any) is the correct one?
- The input layer passes the data directly to the first hidden layer where the data is multiplied by the first hidden layer's weights.
- The input layer passes the data through the activation function before passing it on. The data is then multiplied by the first hidden layer's weights.
- The input layer has its own weights that multiply the incoming data. The input layer then passes the data through the activation function before passing it on. The data is then multiplied by the first hidden layer's weights.
Thanks!
EDIT 1: Here is an image and an example for further clarity.