1

I would like to introduce some bias. I have n-risk factors (predictors) but based on the evidence I collected! I consider one of the Risk factors more relevant than the other ones. There is a weights_column parameter (see description), but it is not clear for me how to use it and if it can be used for my purpose.

The documentation states (version 3.20.0.1):

This option specifies the column in a training frame to be used when determining weights. Weights are per-row observation weights and do not increase the size of the data frame. This is typically the number of times a row is repeated, but non-integer values are also supported. During training, rows with higher weights matter more, due to the larger loss function pre-factor.

I don't know if it is suitable for my purpose. I have the following questions/comments, that would help me to understand how to use it, but also for improving the documentation in next version:

  1. "The weights are per row observation", I was expecting by column (we are defining a weight in a specific column). It seems to me that the algorithm is adding dummy rows, but then it says that is not increasing the number of rows. i) What is the logic for adding such fictitious rows? (Is it a copy of another row or a modified row, if so how it's modified?)

  2. "Due to the larger loss function pre-factor": ii) What does it mean in this context? ii) Does it apply only for loss-function metric? iv) What is the pre-factor?

  3. v) Can we specify more than one column? The name of the parameter is in plural, but the example and the documentation, seem to refer to just one column.

Then the next paragraph says: "For example, a weight of 2 is identical to duplicating a row", but the user can only specify the column name. vi) Can we specify the weight factor number?

The example provided in the documentation does not clarify the purpose of using such parameter based on the problem nature and there is no comparison on how the result may be affected by the use of such parameter. vii) What is the rationale in this case for setting this parameter with the column weight?

For example, running the script with and without setting weights_column we get:

[1] "AUC with weights_column"
[1] 0.9645522
[1] "AUC without weights_column"
[1] 0.9803922

The example shows how to use the argument, but it is not suitable to see the benefit of using it.

1 Answer 1

1

I think there may be some confusion here over the general use of weights.

Here the weighting is applied on a row basis, so cannot be used to up-weight a variable. An example use case might be if we have time-varying data, where we believe that the most recent data is more like future samples than previous data. In this use case, we could up-weight the most recent samples and down weight less recent samples.

The important point here is that it is a weighting of a sample (and all of its features), not of an individual feature across all samples.

For your use case, if you have sufficient training data, it is likely that an appropriate algorithm (GBM/RF) will make use of the feature that you consider to be important. If it does not then this may indicate that the feature you have identified is not as important as you first thought or is highly correlated with another feature.

If you still want to up-weight a feature then a hacky approach to this is to add multiple dummy variables to the data frame for the same feature.

2
  • as you said, it seems to be more suitable for weighting observations for time-varying data than for a particular feature. The documentation does not help too much. I was thinking like in a linear regression model, where the weights quantify the importance of each feature, but this is what we are looking when we build the model. h2o.varimp for GLM informs about coefficients and sign (a proxy of importance as it is in GBM/RF). I get slightly better results using GLM.
    – David Leal
    Jun 15, 2018 at 14:40
  • 1
    The important point here is that you are extracting them from a fitted model not pre-specifying them. You might get better performance if you tune some parameters. In particular look at increasing the number of trees and adding early stopping for the gbm and rf models.
    – Sam Abbott
    Jun 19, 2018 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.