0

I am testing an SVM with a sigmoid kernel on the iris data using sklearn and SVC. Its performance is extremely poor with an accuracy of 25 %. I'm using exactly the same code and normalizing the features as https://towardsdatascience.com/a-guide-to-svm-parameter-tuning-8bfe6b8a452c (sigmoid section) which should increase performance substantially. However, I am not able to reproduce his results and the accuracy only increases to 33 %.

Using other kernels (e.g linear kernel) produces good results (accuracy of 82 %). Could there be an issue within the SVC(kernel = 'sigmoid') function?

Python code to reproduce problem:


##sigmoid iris example
from sklearn import datasets 
iris = datasets.load_iris()
from sklearn.svm import SVC 

sepal_length = iris.data[:,0] 
sepal_width = iris.data[:,1]

#assessing performance of sigmoid SVM
clf = SVC(kernel='sigmoid') 
clf.fit(np.c_[sepal_length, sepal_width], iris.target) 
pr=clf.predict(np.c_[sepal_length, sepal_width])
pd.DataFrame(classification_report(iris.target, pr, output_dict=True))

from sklearn.metrics.pairwise import sigmoid_kernel 
sigmoid_kernel(np.c_[sepal_length, sepal_width]) 

#normalizing features
from sklearn.preprocessing import normalize 
sepal_length_norm = normalize(sepal_length.reshape(1, -1))[0] 
sepal_width_norm = normalize(sepal_width.reshape(1, -1))[0] 
clf.fit(np.c_[sepal_length_norm, sepal_width_norm], iris.target) 
sigmoid_kernel(np.c_[sepal_length_norm, sepal_width_norm]) 

#assessing perfomance of sigmoid SVM with normalized features
pr_norm=clf.predict(np.c_[sepal_length_norm, sepal_width_norm])
pd.DataFrame(classification_report(iris.target, pr_norm, output_dict=True))


0

1 Answer 1

3

I see what's happening. In sklearn releases pre 0.22 the default gamma parameter passed to the SVC was "auto", and in subsequent releases this was changed to "scale". The author of the article seems to have been using a previous version and therefore implicitly passing gamma="auto" (he mentions that the "current default setting for gamma is ‘auto’"). So if you're on the latest release of sklearn (0.23.2), you'll want to explicitly pass gamma='auto' when instantiating the SVC:

clf = SVC(kernel='sigmoid',gamma='auto') 
#normalizing features
sepal_length_norm = normalize(sepal_length.reshape(1, -1))[0] 
sepal_width_norm = normalize(sepal_width.reshape(1, -1))[0] 

clf.fit(np.c_[sepal_length_norm, sepal_width_norm], iris.target)

enter image description here

So now when you print the classification report:

pr_norm=clf.predict(np.c_[sepal_length_norm, sepal_width_norm])
print(pd.DataFrame(classification_report(iris.target, pr_norm, output_dict=True)))

#                    0          1          2  accuracy   macro avg  weighted avg
# precision   0.907407   0.650000   0.750000  0.766667    0.769136      0.769136
# recall      0.980000   0.780000   0.540000  0.766667    0.766667      0.766667
# f1-score    0.942308   0.709091   0.627907  0.766667    0.759769      0.759769
# support    50.000000  50.000000  50.000000  0.766667  150.000000    150.000000

What would explain the 33% accuracy you were seeing is the fact that the default gamma is "scale", which then places all predictions in a single region of the decision plane, and as the targets are split into thirds you get a maximum accuracy of 33.3%:

clf = SVC(kernel='sigmoid') 
#normalizing features
sepal_length_norm = normalize(sepal_length.reshape(1, -1))[0] 
sepal_width_norm = normalize(sepal_width.reshape(1, -1))[0] 

clf.fit(np.c_[sepal_length_norm, sepal_width_norm], iris.target) 

X = np.c_[sepal_length_norm, sepal_width_norm]

enter image description here

pr_norm=clf.predict(np.c_[sepal_length_norm, sepal_width_norm])
print(pd.DataFrame(classification_report(iris.target, pr_norm, output_dict=True)))

#               0     1          2  accuracy   macro avg  weighted avg
# precision   0.0   0.0   0.333333  0.333333    0.111111      0.111111
# recall      0.0   0.0   1.000000  0.333333    0.333333      0.333333
# f1-score    0.0   0.0   0.500000  0.333333    0.166667      0.166667
# support    50.0  50.0  50.000000  0.333333  150.000000    150.000000

1
  • Thanks a lot for the elaborate answer!
    – mar_mor97
    Commented Dec 9, 2020 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.