8

So one of the standard things to do with the data is normalize it and standardize it to have data that's normally distributed with a mean 0 and standard deviation of 1, right? But, what if the data is NOT normally distributed?

Also, does the desired output has to be normally distributed too? What if I want my feedforward net to classify between two classes (-1, and 1), that would be impossible to standardize into a normal distribution of mean 0 and std of 1 right?

Feedforward nets are non-parametric right? So if they are, is it still important to standarize data? And why do people standarize it?

1

1 Answer 1

15

Standardizing the features isn't to make the data fit a normal distribution, it's to put the feature values in a known range that makes it easier for algorithms to learn from the data. This is because most algorithms are not scale/shift invariant. Decision Trees, for example, are both scale and shift invariant, and so doing the normalization has no impact on the performance of the tree.

Also, does the desired output has to be normally distributed too?

No. That's not a thing. The output is whatever the output is. You do have to make sure the activation function of the final layer of your network can make the predictions you want (i.e.: Sigmoid activation can't output negative values or values > 1).

Feedforward nets are non-parametric right?

No, they would generally be considered parametric. Parametric / non-parametric doesn't really have a hard definition. People may mean slightly different things when talking about this.

So if they are, is it still important to standardize data?

Those things have nothing to do with each other at all.

And why do people standardize it?

That's the very first thing I mention, it's to make learning easier/possible.

2
  • Thanks for your reply. On your point you mentioned the use of standardization to make sure inputs fall within a specific range. But I have read some people use standardization to make sure the inputs/outputs have mean 0 and a uniform variance (even if they already are within a range). Why?
    – Luis Cruz
    Oct 14, 2015 at 19:51
  • 1
    I just answered why. Its to make optimization/learning easier. Or to put the output in the range that the activation function can actually reach. Oct 14, 2015 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.