What is the kernel trick in SVM?
A kernel computes the dot product of two points as if they were mapped to a higher-dimensional space, without actually building that space. It lets a linear SVM draw curved boundaries cheaply. Common kernels: linear, polynomial, RBF.
Explain it simply
On a single line of numbers you cannot separate the middle group from the two outer groups with one cut. But lift each point up into a bowl-shaped curve and suddenly one flat horizontal cut does the job.
The kernel trick gives you the benefit of that lift without computing the lifted coordinates.
Example
1-D data: class A at -1, 0, 1 and class B at -3, -2, 2, 3. No single threshold separates them.
Map each x to (x, x^2):
A: (-1, 1) (0, 0) (1, 1) x^2 is at most 1
B: (-3, 9) (-2, 4) (2, 4) (3, 9) x^2 is at least 4Now the horizontal line x^2 = 2.5 separates them perfectly.
A polynomial or RBF kernel does this implicitly. The RBF kernel is K(a, b) = exp(-gamma x ||a - b||^2), a similarity score near 1 for close points and near 0 for far ones.
from sklearn.svm import SVC
SVC(kernel="rbf", gamma="scale") # curved boundary
SVC(kernel="linear") # straight boundaryWhat to say in the interview
"The kernel trick replaces dot products with a kernel function that equals a dot product in a higher-dimensional space, so the SVM can learn non-linear boundaries without computing that space. RBF is a good default; linear is better for very high-dimensional sparse data such as text."
Common mistakes
- Using RBF without scaling the inputs.
- Assuming a kernel always helps; a linear kernel can be enough and is faster.