5

I am new to Machine Learning 7 I have started following Udacity's Intro to Machine Learning

I was following Simple Vector Machine's when this concept of C and Gamma came along. I did some digging around and found the following:

C - A high C tries to minimize the misclassification of training data and a low value tries to maintain a smooth classification. This makes sense to me.

Gamma - I am unable to understand this one.

Can someone explain this to me in layman terms?

1

3 Answers 3

7

When you are using SVM, you are necessarily using one of the kernels: linear, polynomial or RBF=Radial Base Function (also called Gaussian Kernel) or anything else . The latter is

K(x,x') = exp(-gamma * ||x-x'||^2) 

which explicitly contains your gamma. The larger the gamma, the narrower the gaussian "bell" is.

I believe, as you go with the course, you will learn more about such "kernel trick".

1
  • 1
    While your answer is the accepted answer, you do not really explain in layman's terms like OP requested. Dec 9, 2021 at 6:50
7

Intuitively, the gamma parameter defines how far the influence of a single training example reaches, with low values meaning ‘far’ and high values meaning ‘close’. The gamma parameters can be seen as the inverse of the radius of influence of samples selected by the model as support vectors. The C parameter trades off misclassification of training examples against simplicity of the decision surface. A low C makes the decision surface smooth, while a high C aims at classifying all training examples correctly by giving the model freedom to select more samples as support vectors. http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html

6

-C parameter: C determines how many data samples are allowed to be placed in different classes. If the value of C is set to a low value, the probability of the outliers is increased, and the general decision boundary is found. If the value of C is set high, the decision boundary is found more carefully.

C is used in the soft margin, which requires understanding of slack variables.

-Soft margin classifier:
Text

-slack variables Text determine how much margin to adjust.

gamma parameter: gamma determines the distance a single data sample exerts influence. That is, the gamma parameter can be said to adjust the curvature of the decision boundary.

Not the answer you're looking for? Browse other questions tagged or ask your own question.