Questions tagged [h2o]
Use this tag for questions about the H2O in-memory machine learning platform. Where relevant, add language tags like [r], [python], [scala], or [java].
1,879
questions
0
votes
1
answer
12
views
Recursive Feature Elimination with H2O Random Forest
I am using h2o package in python to build a fairly complex model.
It has around 1500 features, but I know that most of them are not really important and I would like to extract the subset of a given ...
0
votes
0
answers
43
views
How to set size of total memory for sparkling water cluster in Databricks
I am working in Databricks with Sparkling Water 3.40.0.4; I have a total driver memory of 512 GB and six workers with 64 GB each. When I call
hc = H2OContext.getOrCreate()
the internal H2O cluster is ...
0
votes
0
answers
7
views
DALEX error : Error in contribution[nrow(contribution), ] <- cumulative[nrow(contribution), : incorrect number of subscripts on matrix
Trying to use DALEX on my data. Getting following error in line pb_h2o_automl <- predict_parts(explainer_h2o_automl,new_observation = new_date_birth,type="break_down")
Error
Error in ...
0
votes
1
answer
27
views
Meaning of predValue field in H2O Random forest model
I built an H2ORandomForestEstimator model using train() method on my spark dataframe with the target column containing values 0 or 1. I downloaded and printed its mojo files using model.download_mojo(...
0
votes
2
answers
49
views
How to fix "Actual column must contain binary class labels, but found cardinality" error in R when fitting the h2o model?
I am getting an error. It occurs when I use the deep learning function in h2o in R. My response is a categorical variable which takes on 3 values so I can't change it to binary labels.
Error: java....
1
vote
0
answers
81
views
I can't run h2o cluster in R
I can't seem to get h2o running in R.
I can easily import h2o using library(h2o) which displays
----------------------------------------------------------------------
Your next step is to start H2O:
...
0
votes
0
answers
40
views
Single column data in h2o model needs 2 columns to work
Having had AutoML() recommend a model after I experimented with a single column of data and needed to identify a target as a step-1 of the single data to train the successful model I now want to use ...
0
votes
0
answers
16
views
When training a neural network with h2o, how do get the history of the reconstruction error?
I use h2o in R to create a neural network, let's say it is an autoencoder. How do get the history of the prediction error, meaning how it evolved over training?
library(h2o)
h2o.init()
nn <- h2o....
0
votes
1
answer
2k
views
Loading checkpoint shards takes too long
I'm very new to generative AI. I have 64gb RAM and 20GB GPU. I used some opensource model from Huggingface and used Python to simply prompt it with out of box model and displaying the result. I ...
1
vote
1
answer
41
views
How does h2o's distributed random forest handle multi-class problems?
The documentation for distributed random forest in h2o states that, for multiclass problems, "a tree is used to estimate the probability of each class separately". I can see this in ...
0
votes
1
answer
38
views
H2O build failed with error "cannot find symbol" in h2o-algos
I tried to build H2O open source, nomatter stable version or clone version, build process via ./gradlew build are failed due to "error: cannot find symbol", the detailed error info as below:
...
0
votes
1
answer
27
views
Facing issues with H2O MOJO deployment using sparkling water on Google Dataproc
I am trying to migrate a model from Hadoop to GCP. Model MOJO will not be retrained. I am running the model in Dataproc using Airflow spark submit.
Source data format matches with Hadoop source and ...
0
votes
1
answer
20
views
h2o and coxph ERRR on field: _train: Training data must have at least 2 features (incl. response)
I am trying to build a model of Cox Proportional Hazard with h2o. In fact a I have succesfully built a model using deeplearning about survival in gastric cancer dataset. When trying to do it with h2o....
0
votes
1
answer
171
views
H2oGPT - cannot import name 'set_seed'
I'm stuck with H2oGPT, I can't let it run.
I am running on a Windows PC with
11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz 2.30 GHz
32GB memory 64bit
GPU Nvidia GeForce RTX 3050
I followed all ...
0
votes
1
answer
28
views
h2o GBM checkpointing error when using h2o automl for the base model
I want to retrain my h2o model on a new set of observations using checkpoint but facing errors. My code is failing on the train step when using checkpoint. My original model is created using h2o ...
0
votes
1
answer
22
views
h2o H2OGenericEstimator training function not working
I want to enable incremental training using H2O models. I save the trained model using MOJO format for existing dataset/observations. Upon receiving new observations, I would like to load the MOJO-...
-1
votes
1
answer
19
views
H2O Decision Tree
I'm watching an H2O demo, but one of the code lines are cut off, and I have no clue what is supposed to be in the rest.
the gv_call assignment is cut off. Does anyone know what is supposed to be in ...
0
votes
1
answer
26
views
H20 cluster - generate the values in a tabular format for pdp plots
I am using the h20 cluster to train the model using tuned random forest and plot pdp plots using the below code
tx = tuned_rf.explain(test_data, top_n_features=5, include_explanations='pdp')
This is ...
0
votes
1
answer
14
views
how to use initial_weights in h2o from R, when I use it I am getting java.lang.NullPointerException error
I want to do deeplearning using h2o with custom initial weights, if I do not add the initial_weights parameter the code runs fine, but if I add it, the error is keep showing.
wt1 = as.h2o(weight1)
...
0
votes
0
answers
22
views
Error When trying to run sparkling-shell on windows
I have installed sparkling water for apache spark 3.2.3 and trying to run sparkling-shell command on cmd
I also installed "spark3.2.3-bin-hadoop3.2" from the apache spark archive which is ...
-1
votes
1
answer
31
views
How to group by ID numbers in dataset when running H2O automl
I'm using the H2O package for machine learning. The goal is to use my dataset to estimate new predicted ages for each participant in my dataset. My setup for the code was to get rid of all unnecessary ...
0
votes
1
answer
32
views
Deep learning in H2O4GPU
Good day! I had initially built a deep learning estimator on the h2o library in R, and I was thinking of using the h2o4gpu library instead to build it in order to expedite the process. Does anyone ...
1
vote
1
answer
42
views
Are the results of training a H2O autoencoder on the same data and same hyperparameters supposed to always be the same?
Good day! I was training a h2o autoencoder and everytime I did so, the anomaly results (reconstruction score) of the variables I received would always be different, although the data I am using was ...
0
votes
0
answers
20
views
How to re-index nodes based on (a) BFS, or (b) DFS using XSLT?
I have created a random forest model with H2o and exported as PMML.
Below is a snippet showing the first decision tree with nodes and their IDs.
[...]
<TreeModel ...
0
votes
1
answer
78
views
Time issues with training a H2o Autoencoder
I was wondering if anyone knows if there are any glaring problems with the way my H2o autoencoder is being trained that could cause it to take so long? Or if anyone knows of any way I can reduce the ...
1
vote
1
answer
24
views
How to get sample split probability values from tree-based models - esp via the h2o framework
Following the proposed tree interpreter approach (http://blog.datadive.net/interpreting-random-forests/) one can explain a tree-based model prediction using info from the decision path.
I've built ...
0
votes
1
answer
31
views
h2o always returns memory address instead of actual content
My h2o code used to return useful information like printing table and metrics but for some reason now always returns memory address.
Example:
import pandas as pd
import h2o
from h2o.estimators import ...
0
votes
2
answers
51
views
Installing h2o on Databricks under R: jsonlite error message:
I am trying to install h2o (R version) on Databricks.
I started with the following:
%sh
apt-get install libcurl4-openssl-dev -y
apt-get update -y
apt-get install -y r-cran-jsonlite
It all ran fine. ...
0
votes
1
answer
38
views
h2o.init() error ports are not available in R
I just installed h2o (3.40.0.4) in R. I ran the following:
library(h2o)
h2o.init()
and got back the following:
H2O is not running yet, starting it now...
Note: In case of errors look at the ...
0
votes
1
answer
40
views
Unclear about the purpose of using H2O.ai
I am trying to train a neural network with massive amounts of data. I have around 2million rows and 200k columns of data. My data is in a sparse matrix, which I cannot feed into a neural network. I am ...
0
votes
1
answer
152
views
Real time scoring with H2O in docker with initiating h2o once
I am quite new to docker. I am trying to deliver some scores via my h2o models through docker in real time (have not tried this yet: link). Here is my Dockerfile and docker-compose/yml.
FROM python:3....
1
vote
1
answer
29
views
Error when applying h2o::h2o.glrm() to mixed data
I would like reduce the dimensionality of a mixed data set with the help of the h2o.glrm() function from the R package h2o. My data set includes binary variables (nominal variables with two possible ...
0
votes
1
answer
37
views
Calibrated probabilities from mojo object
I have a model developed with h2o 3.38. I calibrated it with a separate data. Now when I score a new data set, I see the calibrated probabilities. If I save the model as:
h2o.save_model(gbm_model, ...
0
votes
1
answer
119
views
Why does my training confusion matrix in h2o AutoML only shows 10k total cases instead of 200k
I am currently using h2o autoML to train a model on a binary classification problem. I have a train (70% ~200k rows), valid (10% ~30k rows), test (10% ~30k rows) and blend (10% ~30k rows) datasets all ...
0
votes
0
answers
28
views
Issue continuing a DeepLearning Model from Checkpoint in H2O-3.38.0.1
I have started to receive the following error several months and several checkpoints into training a model from the same checkpoint script. I am triggering the checkpoint script from R-4.2.1 using the ...
0
votes
0
answers
20
views
H2OAutoML.fit() fails with "Wrong FS" exception when keepBinaryModels=True on Windows
Training works fine with keepBinaryModels=False, but when set True I get:
Traceback (most recent call last):
File "C:\spark\spark-3.2.4-bin-hadoop2.7\python\pyspark\sql\utils.py", line 117,...
1
vote
1
answer
75
views
H2O Variable standardization
The documentation in standardize section https://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/algo-params/standardize.html only includes these algorithms: Deep Learning, GLM, GAM, K-Means.
I ...
0
votes
0
answers
450
views
How to resolve the error "the spark driver has stopped unexpectedly and is restarting" when converting PySpark Dataframe to H2O Frame
I recently started exploring the field of Data Engineering and came across some difficulties. I have a bucket in GCS with millions of parquet files and I want to create an Anomaly Detection model with ...
0
votes
1
answer
44
views
Get Stage Results from H2O Sparkling Water model
I am looking to create a confidence interval for one of my model's outputs and I need to get the model outputs before the link function is applied. From what I've read, it seems like I am interested ...
0
votes
1
answer
83
views
Training on entire dataset in AutoML function of h2o
I am using h2o.automl function in R and here you can find the function below;
h2o.automl(
x = x_name,
y = y_name,
training_frame = as.h2o(train),
leaderboard_frame = as.h2o(test),
...
0
votes
0
answers
31
views
h2o sparkling: Error reading MOJO JSON: Object not supported:
I'm running the Sparkling Water automl example (https://docs.h2o.ai/sparkling-water/3.3/latest-stable/doc/ml/sw_automl.html) on a local spark install on Windows (added code at the top setting up the ...
0
votes
0
answers
38
views
H2o Cluster Resources shared issue while on XGBoosts model process
I have used XGBoost for my model. i have noticed that h2o cluster not share memory while on this model process. master A server RAM utilization is very high and master B RAM utilization is very low. ...
0
votes
1
answer
32
views
H2O UI Hadoop instance after failure (new attempt)
I've started H2O 3 nodes instance on Hadoop (yarn), at night something went wrong at cluster and I see job container attempt 1 was killed and yarn started new job on different node attempt 2. 3 mapper ...
0
votes
0
answers
59
views
leaderboard for auto-pytorch models
I tried several AutoML frameworks such as AutoGluon, H2O AutoML, and Auto-PyTorch. AutoGluon and H2O AutoMLboth have functions producing a leader board (roc_auc, accuracy, log loss, etc.). However, I ...
1
vote
1
answer
296
views
H2O features transformations inside MOJO models
I'm checking how we can use H2O MOJO models to run scoring models at prod using java api. I've received MOJO model in zip file from data scientists teams and managed to produce some predictions. Now I ...
0
votes
1
answer
14
views
How to group by multiple columns in h2o frame?
I have a monthly timeseries dataframe grouped by date and team name. I want to convert it to a h2o frame to run the automl models but the index gets dropped in hf_train. How do I group this data in ...
1
vote
1
answer
107
views
Increasing point size in h2o.shap_summary_plot
I would like to increase point in h2o SHAP summary plot. I tried to modify object that is returned by h2o.shap_summary_plot:
plt=h2o.shap_summary_plot(model, test)
plt=plt+geom_point(size = 4)
...
1
vote
1
answer
143
views
set reference/base level for categorical variable in H2O model
I am experiencing what appears to be strange behavior while trying to set base/reference levels for a categorical variable in a simple glm model using H2O. To illustrate, I have added a few additional ...
0
votes
1
answer
51
views
Is Variable Importance calculation on H2O based on Train set?
I'm pretty sure that h2o.varimp is not based on data other than train or validation since the test data is never put into the model.
I was reading the h2o documents about Variable Importance but ...
1
vote
2
answers
106
views
H2O SplitFrame throws error reached elapsed time limit
I'm using RStudio server and h2o.splitFrame is causing some error, and I'm not sure where to look at.
h2o.init(max_mem_size = '950G', nthreads = 127)
...
parts = h2o.splitFrame(data_h2o, ratios = ...