The dataset has 177927 rows and 820 columns of one-hot encoded features. There is no NaN in the dataset. I want to build two H2O XGBoost models for regression on two kinds of labels ('count_5' and 'count_overlap') respectively, using the same feature matrix. I use python 3.8 on Ubuntu.
'count_5' has 4 unique numeric labels (from 0 to 4).
label | frequency |
---|---|
0 | 159466 |
1 | 18102 |
2 | 346 |
3 | 13 |
'count_overlap' has 2416 unique numeric labels.
label | count_overlap |
---|---|
0 | 53077 |
1 | 9989 |
2 | 5430 |
3 | 3224 |
4 | 2570 |
... | ... |
6558 | 1 |
2257 | 1 |
2385 | 1 |
2204 | 1 |
2047 | 1 |
Here is the main part of code for both models:
# Generate H2O frame
train = h2o.H2OFrame(mydf)
y = label_name
X = list(train.columns)
X.remove(y)
train[y] = train[y].asnumeric()
# Model
estimator = H2OXGBoostEstimator(
seed=1,
distribution="poisson",
model_id='XGB_default',
keep_cross_validation_predictions=True,
keep_cross_validation_fold_assignment=True,
nfolds=2,
)
estimator.train(X, y, train)
# save predictions
y_pred = estimator.cross_validation_holdout_predictions()
y_true = train[y]
y_true_pd = h2o.as_list(y_true)
y_pred_pd = h2o.as_list(y_pred)
# performance
estimator.cross_validation_metrics_summary().as_data_frame()
The H2O XGBoost model on 'count_5' gave reasonable results:
Training: Label: count_5 Model: XGB xgboost Model Build progress: |███████████████████████████████████████████| 100%
mean | sd | cv_1_valid | cv_2_valid | |
---|---|---|---|---|
mae | 0.20095341 | 2.6120833E-4 | 0.20076871 | 0.20113811 |
mean_residual_deviance | 0.74664176 | 0.0035013587 | 0.74911755 | 0.7441659 |
mse | 0.11081107 | 0.0011397477 | 0.11161699 | 0.11000515 |
r2 | -0.027853519 | 9.893299E-4 | -0.027153956 | -0.02855308 |
residual_deviance | 0.74664176 | 0.0035013587 | 0.74911755 | 0.7441659 |
rmse | 0.33288077 | 0.0017119459 | 0.3340913 | 0.33167022 |
rmsle | 0.22899812 | 5.8065885E-4 | 0.22940871 | 0.22858754 |
Scoring History:
timestamp | duration | number_of_trees | training_rmse | training_mae | training_deviance |
---|---|---|---|---|---|
2021-01-13 13:35:09 | 15.256 sec | 0.0 | 0.506659 | 0.503162 | 1.158219 |
2021-01-13 13:35:12 | 18.632 sec | 1.0 | 0.433015 | 0.422635 | 1.004022 |
2021-01-13 13:35:12 | 18.830 sec | 2.0 | 0.387392 | 0.363154 | 0.899638 |
2021-01-13 13:35:13 | 19.034 sec | 3.0 | 0.360412 | 0.319287 | 0.830496 |
... ... | ... ... | ... ... | ... ... | ... ... | ... ... |
2021-01-13 13:35:15 | 21.244 sec | 14.0 | 0.325060 | 0.203695 | 0.706665 |
2021-01-13 13:35:15 | 21.452 sec | 15.0 | 0.324720 | 0.202657 | 0.704868 |
2021-01-13 13:35:16 | 22.861 sec | 50.0 | 0.311705 | 0.191559 | 0.649280 |
Here are the y_true ('count_5') and y_pred
count_5 | y_pred |
---|---|
0 | 0.098148 |
1 | 0.129788 |
1 | 0.181357 |
0 | 0.037972 |
0 | 0.165198 |
... | ... ... |
0 | 0.156512 |
0 | 0.138887 |
1 | 0.257443 |
0 | 0.077034 |
0 | 0.037227 |
However, the H2O XGBoost model on 'count_overlap' gave NaN predictions without warning or error raised:
Training: Label: count_overlap Model: XGB xgboost Model Build progress: |███████████████████████████████████████████| 100%
mean | sd | cv_1_valid | cv_2_valid | |
---|---|---|---|---|
mae | NaN | 0.0 | NaN | NaN |
mean_residual_deviance | NaN | 0.0 | NaN | NaN |
mse | NaN | 0.0 | NaN | NaN |
r2 | NaN | 0.0 | NaN | NaN |
residual_deviance | NaN | 0.0 | NaN | NaN |
rmse | NaN | 0.0 | NaN | NaN |
rmsle | NaN | 0.0 | NaN | NaN |
timestamp | duration | number_of_trees | training_rmse | training_mae | training_deviance |
---|---|---|---|---|---|
2021-01-13 17:04:44 | 12.047 sec | 0.0 | 415.741082 | 110.880732 | 154.986121 |
2021-01-13 17:04:47 | 15.042 sec | 1.0 | inf | inf | NaN |
Here are the y_true ('count_overlap') and y_pred:
count_overlap | y_pred |
---|---|
0 | NaN |
1247 | NaN |
960 | NaN |
0 | NaN |
39 | NaN |
... | ... ... |
24 | NaN |
0 | NaN |
540 | NaN |
0 | NaN |
57 | NaN |
The questions are:
H2O XGBoost did quite well for the 'count_5' label. I also tried other H2O models. Random Forest, SVM, Deep Learning, and GLM all gave good results for both labels (no NaN at all). Why H2O XGBoost predicted NaN 'count_overlap' label? Is there any suggestion or solution?
Any help would be appreciated!!