15

Is regularisation a subset of normalisation ? I know normalisation is used when all the values are not on the same scale, but normalisation is also used to tone down the values, and so is regularisation .So what's the difference between two?

1

1 Answer 1

35

Normalisation adjusts the data; regularisation adjusts the prediction function.

As you noted, if your data are on very different scales (esp. low-to-high range), you likely want to normalise the data: alter each column to have the same (or compatible) basic statistics, such as standard deviation and mean. This is helpful to keep your fitting parameters on a scale that the computer can handle without a damaging loss of accuracy.

One goal of model training is to identify the signal (important features) and ignore the noise (random variation not really related to classification). If you give your model free rein to minimize the error on the given data, you can suffer from overfitting: the model insists on predicting the data set exactly, including those random variations.

Regularisation imposes some control on this by rewarding simpler fitting functions over complex ones. For instance, it can promote that a simple log function with a RMS error of x is preferable to a 15th-degree polynomial with an error of x/2. Tuning the trade-off is up to the model developer: if you know that your data are reasonably smooth in reality, you can look at the output functions and fitting errors, and choose your own balance.

2
  • 1
    Got it . Thanks !
    – harshi
    Oct 30, 2017 at 19:09
  • 2
    @harshi If this answered your question, you should select it as an answer. Otherwise, the question remains open, and others won't find it as readily when they have a similar query. Mar 12, 2020 at 21:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.