I'm trying to deploy my different pipelines all in one with features union, everything works except one problem.
In my DataFrame I have a column ID, that I want to keep untouched through all the pipeline. I have to give it to the pipeline because I apply some one hot encode and other stuff, I cannot just merge it back at the end.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
scaler_pipeline = Pipeline([
('selector', DataFrameSelector(col_scalar)),
('imputer', SimpleImputer(strategy="median")),
('std_scaler', StandardScaler())
])
one_hot_pipeline = Pipeline([
('selector', DataFrameSelector(col_one_hot)),
('imputer', SimpleImputer(strategy="most_frequent")),
('one_hot', OneHotEncoder())
])
full_pipeline = FeatureUnion(transformer_list=[
("DataFrameSelector", DataFrameSelector(immutable_col)),
("scaler_pipeline", scaler_pipeline),
("one_hot_pipeline", one_hot_pipeline),
])
Where my DataFrameSelector is just this:
class DataFrameSelector(BaseEstimator, TransformerMixin):
def __init__(self, attribute_names):
self.attribute_names = attribute_names
def fit(self, X, y=None):
return self
def transform(self, X):
return X[self.attribute_names]
At the beginning of the "full_pipeline" I want to select some columns (the ID here) and just keep it without touching it.
For now I get this error
TypeError: no supported conversion for types: (dtype('O'), dtype('float64'), dtype('float64'))