1

I am trying to generate a model that uses several physico-chemical properties of a molecule (incl. number of atoms, number of rings, volume, etc.) to predict a numeric value Y. I would like to use PLS Regression, and I understand that standardization is very important here. I am programming in Python, using scikit-learn. The type and range for the features varies. Some are int64 while other are float. Some features generally have small (positive or negative) values, while other have very large value. I have tried using various scalers (e.g. standard scaler, normalize, minmax scaler, etc.). Yet, the R2/Q2 are still low. I have a few questions:

  1. Is it possible that by scaling, some of the very important features lose their significance, and thus contribute less to explaining the variance of the response variable?
  2. If yes, if I identify some important features (by expert knowledge), is it OK to scale other features but those? Or scale the important features only?
  3. Some of the features, although not always correlated, have values that are in a similar range (e.g. 100-400), compared to others (e.g. -1 to 10). Is it possible to scale only a specific group of features that are within the same range?

2 Answers 2

2

The whole idea of scaling is to make models more robust to analysis on features space. For example, if you have 2 features as 5 Kg and 5000 gm, we know both are same, but for some algorithm, which are sensitive to metric space such as KNN, PCA etc, they will be more weighted towards second features, so scaling must be done for these algos.

Now coming to your question,

  1. Scaling doesn't effect the significance of features. As i explained above, it helps in better analysis of data.
  2. No, you should not do, reason explained above.
  3. If you want to include domain knowledge in your model, you can use it as prior information. In short, for linear model, this is same as regularization. It has very good features. if you think, you have many useless-features, you can use L1 regularization, which creates sparse effect on features space, which is nothing but assign 0 weight to useless features. Here is the link for more-info.

One more point, some method such as tree based model doesn't need scaling, In last, it mostly depend on the model, you choose.

4
  • Thank you for your answers and suggestions. I heard of block scaling (see 3), but I was really confused about the idea.
    – Yann
    Jan 15, 2019 at 21:02
  • In addition to this, I am using scikit-learn, which provides a class for PLSRegression. That class implements the NIPALS algorithm, and each object provides a scaler. So basically, you could scale internally. I have not figured out what is the exact type of scaling that is used here. My question is, does it make sense to enable internal scaling of my training set, after I have already scaled the data externally with a custom standardizer?
    – Yann
    Jan 17, 2019 at 17:32
  • Some algo prefer different scaling method, let it do its job. For exp, you used std_scaler, now data will have mean of 0 and var of 1. So it is gaussian distribution with negative values too. But some algo prefer min-max, which has data in range (0,1), it will scaled it in desired form. In last, it will not affect data anyway. Jan 17, 2019 at 17:52
  • Thank you. I understand this. I have compared R2/Q2 in some examples, and it does not necessarily help to have both scaling operations turned on.
    – Yann
    Jan 18, 2019 at 13:59
1
  1. Lose significance? Yes. Contribute less? No.
  2. No, it's not OK. It's either all or nothing.
  3. No. The idea of scaling is not to decrease / increase significance / effect of a variable. It's to transform all variables to a common scale that can be interpreted.
1
  • Thanks for your answer.
    – Yann
    Jan 15, 2019 at 21:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.