2

I am trying to use SVR with an rbf kernel (obviously) on a regression problem. My dataset has something like 300 features. I would like to select more relevant features and use something like the sequentialfs function of matlab which would try every combination (or anyway starting with few variables and adding variables on the way, or the opposite, going backward, like the RFE or RFECV of scikit)).

Now, as said, for python there is the RFE but I can't use it with a non linear estimator. Is there anything I am not aware about which would allow me to use the RFE with a non linear estimator? I suppose it wouldn't be difficult to write an algorithm that does that, but if there is already something which does the job it would be definitely better.

Thank you as always ;)

1 Answer 1

1

RFE must have some way of determining the feature importances and RBF kernel SVMs don't provide that. From a quick glance at the original algorithm, I get the impression that an implementation for kernel SVMs would have to integrated into the SVM learning algorithm.

Stepwise (forward or backward) feature selection would be easy to implement, but is not available as a pre-built estimator in scikit-learn.

2
  • Thank again you larsmans. What do you mean when you say "From a quick glance at the original algorithm, I get the impression that an implementation for kernel SVMs would have to integrated into the SVM learning algorithm." I didn't get that part.
    – foebu
    Mar 9, 2014 at 16:16
  • @foebu: did you read the paper? Section 6.3 discusses the kernelized case. The RFE algorithm is specialized to deal with this, i.e. it becomes a different algorithm that has knowledge of how SVMs work.
    – Fred Foo
    Mar 9, 2014 at 22:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.