How does SMOTE work, and what are its limitations?
SMOTE creates synthetic minority samples by picking a minority point, choosing one of its nearest minority neighbours, and placing a new point at a random spot on the line between them. It reduces duplication but can create noisy points near class borders.
Explain it simply
Plain oversampling photocopies the same minority rows, so a model can memorise them. SMOTE (Synthetic Minority Over-sampling Technique) instead makes new, slightly different examples that lie between existing ones.
Picture two defaulters on a map of income versus loan size. SMOTE drops a new "defaulter" somewhere on the straight line joining them.
Example
Formula: new point = x + lambda x (neighbour - x), with lambda random between 0 and 1.
x = (income 2, loan 4)
neighbour = (income 4, loan 8)
lambda = 0.5
new point = (2, 4) + 0.5 x (2, 4) = (3, 6)Using the imbalanced-learn library:
from imblearn.over_sampling import SMOTE
X_res, y_res = SMOTE(k_neighbors=5, random_state=0).fit_resample(X_train, y_train)Limits:
- It works on numeric features; categorical columns need
SMOTENC. - If minority and majority overlap, synthetic points can land inside the majority region and add noise.
- It needs distances, so scale features first.
- Apply it to the training split only, ideally inside an imblearn
Pipelineso each cross-validation fold is handled properly.
What to say in the interview
"SMOTE interpolates between a minority sample and its nearest minority neighbours to create synthetic rows. It beats duplication but can blur class boundaries and does not suit categorical data without SMOTENC. I apply it only on training folds and compare it against class weights and threshold tuning."
Common mistakes
- Running SMOTE before cross-validation, which leaks synthetic copies into validation folds.
- Using it on unscaled features.