What do the C and gamma parameters do in an RBF SVM?
C controls how much misclassification is tolerated: high C fits training data tightly (risk overfitting), low C allows a wider, smoother margin. Gamma sets how far one point's influence reaches: high gamma gives wiggly, local boundaries, low gamma gives smooth ones.
Explain it simply
C is the strictness of a teacher. A strict teacher (high C) wants every student seated exactly in the right row and will redraw the seating plan to fit each exception. A relaxed teacher (low C) accepts a few students in the wrong row for a tidy plan.
Gamma is the reach of each student's voice. High gamma: only immediate neighbours hear it, so the boundary bends around each point. Low gamma: the voice carries across the room, so the boundary is smooth.
Example
Low C, low gamma : very smooth, may underfit
High C, high gamma : bends around every point, likely overfits
Low C, high gamma : still fairly noisy
High C, low gamma : near-linear but strictTypical tuning, searching on a logarithmic grid:
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
pipe = make_pipeline(StandardScaler(), SVC(kernel="rbf"))
grid = GridSearchCV(pipe,
{"svc__C": [0.1, 1, 10, 100],
"svc__gamma": [0.001, 0.01, 0.1, 1]}, cv=5)That grid has 4 x 4 = 16 combinations, and with cv=5 it trains 16 x 5 = 80 models.
What to say in the interview
"C trades margin width against training errors, and gamma sets the reach of each point in the RBF kernel. Too high on either overfits. I scale the data and tune both together with cross-validated grid search on a logarithmic scale."
Common mistakes
- Tuning C and gamma separately.
- Searching on a linear grid like 1, 2, 3 instead of powers of ten.