22

Say that I want to train BaggingClassifier that uses DecisionTreeClassifier:

dt = DecisionTreeClassifier(max_depth = 1)
bc = BaggingClassifier(dt, n_estimators = 500, max_samples = 0.5, max_features = 0.5)
bc = bc.fit(X_train, y_train)

I would like to use GridSearchCV to find the best parameters for both BaggingClassifier and DecisionTreeClassifier (e.g. max_depth from DecisionTreeClassifier and max_samples from BaggingClassifier), what is the syntax for this?

2 Answers 2

29

I found the solution myself:

param_grid = {
    'base_estimator__max_depth' : [1, 2, 3, 4, 5],
    'max_samples' : [0.05, 0.1, 0.2, 0.5]
}

clf = GridSearchCV(BaggingClassifier(DecisionTreeClassifier(),
                                     n_estimators = 100, max_features = 0.5),
                   param_grid, scoring = choosen_scoring)
clf.fit(X_train, y_train)

i.e. saying that max_depth "belongs to" __ the base_estimator, i.e. my DecisionTreeClassifier in this case. This works and returns the correct results.

2
9

If you are using a pipeline then you can extend the accepted answer with something like this (note the double, double underscores):

model = {'model': BaggingClassifier,
         'kwargs': {'base_estimator': DecisionTreeClassifier()}
         'parameters': {
             'name__base_estimator__max_leaf_nodes': [10,20,30]
         }}
pipeline = Pipeline([('name', model['model'](**model['kwargs'])])
cv_model = GridSearchCV(pipeline, param_grid=model['parameters'], cv=cv, scoring=scorer)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.