What is a support vector machine, and what are support vectors and margin?
An SVM finds the separating line (hyperplane) with the widest margin between two classes. Support vectors are the training points closest to that boundary; only they decide where it sits.
Explain it simply
Imagine drawing a lane divider between two groups of students standing in a field. Many dividers would work, but the best one leaves the widest empty corridor on both sides, so a slightly misplaced student does not cross over. SVM finds that widest-corridor divider.
The students standing right at the edge of the corridor are the support vectors. Move any other student and the divider does not change.
Example
For a boundary w.x + b = 0 the margin width equals 2 / ||w||. Maximising the margin means minimising ||w||.
Boundary: x1 + x2 - 5 = 0, so w = (1, 1), b = -5
Distance of point (4, 3) from it:
|1*4 + 1*3 - 5| / sqrt(1^2 + 1^2) = 2 / 1.414 = 1.414
Margin width = 2 / ||w|| = 2 / 1.414 = 1.414(This is only an arithmetic illustration of the formula; a fitted SVM chooses w and b itself.)
from sklearn.svm import SVC
model = SVC(kernel="linear", C=1.0)
model.fit(X_train, y_train)
print(model.support_vectors_.shape) # rows = number of support vectorsBecause only support vectors matter, SVMs are memory-efficient after training and work well when classes are well separated, even with many features.
What to say in the interview
"An SVM picks the hyperplane that maximises the margin to the nearest points of each class, which are the support vectors. A wider margin usually generalises better. With a soft margin and the kernel trick it handles overlapping and non-linear data."
Common mistakes
- Thinking every training point affects the boundary.
- Forgetting to scale features before an SVM.