Questions tagged [mlr3]
mlr3 is the next generation of the mlr package for machine learning in R.
236
questions
0
votes
0
answers
21
views
Benchmarking mlr3 - build models that differ in included predictors?
I have data where the predictors have some natural grouping (some are questionnaire scores, some are biological variables). Is there any way to use the benchmarking in mlr3 to compare models based on ...
0
votes
1
answer
21
views
knn Imputation for features with mlr3 pipelines; tune value for k with nested resampling
I have another question concerning mlr3 pipelines. In my dataset some values are missing, so what I have learned from reading the literature it is best to delete the cases with missing values on the ...
1
vote
1
answer
29
views
How to extract lambda value from mlr3's cv.glmnet learner after benchmarking grid?
I am currently doing a regression using mlr3 lrn('regr.cv_glmnet'). I am doing a benchmark grid to determine whether linear regression vs cross validated lasso works better. By using default values ...
1
vote
2
answers
41
views
Extract weights from fitted regr.nnet object in mlr3
This question is related to the solution provided by @Sebastian for a previous question. It showed how to do repeated training for a regr.nnet learner using a custom (=fixed) resampling strategy and ...
0
votes
0
answers
24
views
Lasso learner classif.cv.glmnet in mlr3: Access the final model with lamda and β coefficients that is used for performance evaluation and prediction?
I am trying to do a lasso regression for a binary classification task in mlr3 using the learner lrn("classif.cv_glmnet").
My goal is to train this learner on the final task and access the ...
1
vote
1
answer
41
views
Multiple runs and interaction terms in mlr3 regr.nnet task
I am trying to port a few didactical examples from packages nnet, neuralnet and ranger to package mlr3. I like the way how mlr3 can handle fitted models, e.g. model evaluation, feature importance or ...
0
votes
1
answer
25
views
Accessing probabilities for PipeOps TwoClass Classif Learners
I am currently working on mlr3shiny. This program utilizes various mlr3 methods on an R shiny UI to make mlr3-models. I am currently unable to properly integrate the learner-objects to work with DALEX ...
0
votes
0
answers
20
views
Accounting for covariates (site effects) with mlr3 machine learning (pipelines)
I have gone through the entire mlr3 book but am unable to find a solution for how to address site effects in my data, as it comes from a multicenter study. I theoretically know about leave-site-out CV,...
0
votes
1
answer
30
views
mlr3 glmnet Repeated CV and Alpha/Lambda Tuning
I am hoping to use mlr3 to build multiple glmnet models (classification, regression, and survival). I was originally going to use the mlr3 associated cv_glmnet learners. However, in reading further, I'...
0
votes
1
answer
37
views
Define parameter1 as 1 - parameter2 using R paradox package
I want to define parameter1 as 1 - parameter2 using paradox package.
That is parameter 1 depends on parameter 2 (depends argument doesn't help here I suppose).
Here is my reserach space:
search_space =...
0
votes
1
answer
37
views
Missing value handling with imputation in a nested resampling procedure such that there is no information bleed from train to test
I am looking through the documentation for the nested resampling procedure in the mlr3tuning package and I do not see any way that the package can handle NA values such that any information bleed ...
0
votes
0
answers
64
views
Bootstrap resampling for stacked/ensemble leaner with mlr3 in R
So I'm trying to generate bootstrapped resamples for an ensemble model which throws an error. This seems to result from the duplication of row_ids; I suppose these duplicate rows should be expected ...
1
vote
1
answer
35
views
Set 3 double parameters in p_db using paradox package
How can I set parameter with say 3 float values.
For example I want to search parameter X for 0.99, 0.98 and 0.97.
For p_dbl there are lower and upper parameters but not values I can use.
For example ...
0
votes
1
answer
26
views
Custom rolling window nested resampling with mlr3
Since last version, mlr3tuning package supports (custom) instantiated resampling in AutoTuner class: https://github.com/mlr-org/mlr3tuning/releases/tag/v0.17.2
I have tried to construct rolling window ...
1
vote
1
answer
40
views
Variable Importance P-Values
Can the importance_pvalues (https://rdrr.io/cran/ranger/man/importance_pvalues.html) command be used via mlr3? In other words, can I indicate that I would like the p-values outputted in my call to the ...
0
votes
1
answer
27
views
extract_inner_fselect_results is NULL with mlr3 Nested Resampling
This question is an extension of the following question: No Model Stored with Mlr3.
I have been performing nested resampling to get an unbiased metric of model performance. If I don't specify ...
1
vote
2
answers
37
views
Apply Models from Nested Resample to Permuted Dataset
I have generated a nested resampling object with the following code:
data<-read.csv("Data.csv", row.names=1)
data$factor<-as.factor(data$factor)
set.seed(123, "L'Ecuyer")
...
1
vote
1
answer
33
views
Predict on holdout set using mlr3
I am using mlr3 package. I set roles of some rows to "holdout" and than trained the model:
library(mlr3)
# train on iris
task = tsk("iris")
task$nrow
task$set_row_roles(130:150, &...
0
votes
2
answers
83
views
No Model Stored with Mlr3
I am performing nested resampling using the following code:
MSvCon<-read.csv("MS v Control Proteomics Final.csv", row.names=1)
MSvCon$Status<-as.factor(MSvCon$Status)
MSvCon[,2:4399]&...
0
votes
1
answer
26
views
RFE Termination Using RMSE with AutoFSelector
To mimic how caret performs RFE and select features that produce the lowest RMSE, it was suggested to use the archive.
I am using AutoFSelector and nested resampling with the following code:
ARMSS<...
0
votes
1
answer
21
views
mlr3 Multiple Measures AutoFSelector
I wanted to inquire about how to modify my code so that I could get multiple performance measures as an output.
My code is the following:
ARMSS<-read.csv("Index ARMSS Proteomics Final.csv&...
0
votes
2
answers
33
views
Remove columns with many NA values using mlr3pipelines
I am trying to remove columns where proportion of NA value are greater than na_cutoff threshold using mlr3pipelines.
Here is my try:
library(mlr3)
library(mlr3pipelines)
task = tsk("iris")
...
1
vote
1
answer
34
views
mlr3 standard deviation for k-fold cross-validation resampling
Anybody know how to extract the standard deviation for a ResampleResult/BenchmarkResult in mlr3?
The implemented metrics seems to be returning only the average value.
measures <- list(
mlr3::msr(&...
0
votes
1
answer
27
views
Apply mlr3 pipes on group by basis
I would like to know is it possible to apply mlr3 Pipe processing on groupBy basis.
For example, from the mlr3pipelines documentation, we can scale predictors with following code:
library(mlr3)
...
0
votes
1
answer
63
views
mlr3 RFE Termination Metric
This may be a naive question but I would like to use recursive feature elimination with a random forest model and wanted to see if I could terminate based on the feature set that gives the smallest ...
1
vote
1
answer
63
views
How do I tune random forest with oob error?
Instead of doing a CV and train the Random Forest multiple times I would like to use the OOB Error as and unbiased estimation of the generalized error.
And for a few data points (in the low thousands),...
0
votes
1
answer
39
views
Month by month rolling CV in mlr3
My goal is to create Resampling using mlr3 package that uses some sort of rolling CV. More concretely, I want to use n months of data in training set (say 6 months) and one month of data in test set.
...
3
votes
2
answers
83
views
How to make Bayesian hyperparameter optimization reproducible using "mbo" tuner?
I would like to use R's mlr3* packages to build ML algos in a reproducible manner. I have tried to use regr.glmboost learner with mbo tuner and run_time terminator. I have played around with the HPO ...
2
votes
1
answer
49
views
Term_evals when finding hyper parameters for XGBoost with #mlr3
I'm new to gradient boosting (XGBoost). I've read the manual for mlr3, and if I understand it right, I want to first tune my hyper parameters. I'm unsure for this how to set term_evals? In the ...
2
votes
1
answer
77
views
mlr3 properly setting up parallelization
Say I have a machine with 32 cores and want to execute a nested CV of 5 outer folds and 3 inner folds as efficiently as possible.
On the outer fold, I benchmark two or more learners, on the inner fold ...
2
votes
2
answers
69
views
Rewriting ParamSet ids from mlr3::paradox()
Let's say I have the following ParamSet object:
my_ps = paradox::ps(
minsplit = p_int(1, 64, logscale = TRUE),
cp = p_dbl(1e-04, 1, logscale = TRUE))
Is it possible to rename minsplit to ...
2
votes
1
answer
69
views
How to save a ranger model in mlr3 without data?
I have created a ranger model using mlr3 library. I saved this model to my machine using following command. The created file is huge in size. The saved file also has the data along with the model. Is ...
0
votes
0
answers
32
views
CI for ROC anaylsis with independent training and test data using SVM in mlr3
I would like to compute confidence intervalls for ROC analysis using SVM classification with the mlr3 framework:
mlr3_dat.train <- cell_culture
mlr3_dat.test <- human_data
mlr3_task <- ...
0
votes
0
answers
59
views
Why is rfe not working with resample in mlr3 R?
I am currently making an RFE thanks to the mlr3 package following this methodology: https://doi.org/10.3390/rs14215381.
When I use the resample function to optimize the random forest parameters based ...
0
votes
1
answer
76
views
Customized loss function for Random Forest for Regression in R
I want to implement my own customized loss function for a random forest regression in R. I found this Random Forests with a Customized Loss Function on how to do it in python, however not in R.
I ...
0
votes
0
answers
33
views
Is "insample" in mlr3tuning resampling can be used when we want to do hyperparameter tuning with the full dataset?
I've been trying to do some tuning hyperparameters for the survival SVM model. I used the AutoTuner function from the mlr3tuning package. I want to do tuning for the whole dataset (No train & test ...
0
votes
1
answer
131
views
How to Install mlr3extralearners in R?
So I am going to do some survivalsvm process and I need the mlr3extralearners package from GitHub. When I tried to install it locally using the tar.gz file, it said
Execution halted
Warning in install....
1
vote
0
answers
98
views
get coefficients and features from a glmnet learner #mlr3
Thanks for providing the mlr3 package in R , since I am trying it for the first time
simple code in mlr3:
learner = lrn("classif.cv_glmnet")
lrn_glmnet <- learner$train(task, row_ids = ...
0
votes
1
answer
70
views
mlr3 benchmarking with elapsed time measure
I am using the mlr3 package in R to create several classification learners and benchmark them on the same binary classification task. I want to evaluate the learners with multiple performance measures:...
0
votes
0
answers
32
views
`po('imputelearner')` generates an extra level `factor`
After factor encoding imputation, I am expecting to have a feature with 2 levels but I am getting an extra level named 'factor' that no observation has it.
See example below, I really don't know if it'...
0
votes
1
answer
91
views
Extraction of tuned hyperparameters from tuning instance archive
I've build an automated machine learning system based on the following example:
https://mlr-org.com/gallery/2021-03-11-practical-tuning-series-build-an-automated-machine-learning-system/
I've used the ...
0
votes
2
answers
118
views
Linear Regression in mlr3 with Interactions / Quadratic Terms
I try to fit a linear model with interactions and/or quadratic terms in mlr3 benchmark. Unfortunately, I didn't find a possibility on github or stackexchange. Here is an example:
library(mlr3verse)
...
1
vote
1
answer
128
views
How to obtain out of sample predictions on new data with mlr3 resample stored models?
I want to use mlr3 for cross-fitting of nuisance parameters in a semi-parametric model such as TMLE or AIPW. The cross-fitting procedure is similar to k-fold cross-validation; split the data into K ...
2
votes
1
answer
130
views
Setting `early_stopping_rounds` in xgboost learner using mlr3
I want to tune an xgboost learner and set the parameter early_stopping_rounds to 10% of the parameter nrounds (whichever is generated each time).
Should be a simple thing to do in general (i.e. tuning ...
1
vote
1
answer
71
views
Imputation of target using mlr3
After studying the sources describing mlr3 and looking at the given examples I still couldn't find any answer about how to impute the target variable during a regression task, when it has missings. I ...
2
votes
1
answer
90
views
How do I set the upper range mtry tuning value in mlr3, when I also conduct automated feature selection?
Date: 2022-08-17. R Version: 4.0.3. Platform: x86_64-apple-darwin17.0 (64-bit)
Problem: In mlr3 (classif.task, learner: random forest), I use automated hyperparameter optimization (HPO; mtry in the ...
1
vote
1
answer
62
views
How to set the graph learner id in mlr3pipelines?
I construct a benchmark with 4 graph learners on 1 dataset. The learner_id of the result of the benchmark is so long because I have some preprocessings. How can I set the learner id so that it wouldn'...
1
vote
1
answer
131
views
How to set "budget" tag for xgboost hyperband optimization with mlr3tuningspaces?
I am trying to tune xgboost with hyperband and I would like to use the suggested default tuning space from the mlr3tuningspaces package. However, I don't find how to tag a hyperparameter with "...
0
votes
1
answer
110
views
How to define the classification threshold as a (hyper)parameter of a learner for tuning in mlr3 package in R?
there is a function to tune threshold for say a binary classification described here: https://mlr3pipelines.mlr-org.com/reference/mlr_pipeops_tunethreshold.html
Here's my failed attempt:
RF_lrn <-...
0
votes
1
answer
102
views
survxai explainer with an mlr3proba model
I am trying to build a survxai explainer from a survival model built with mlr3proba. I'm having trouble creating the predict_function necessary for the explainer. Has anyone ever tried to build ...