How do class weights, oversampling and undersampling differ for imbalanced data?
Class weights make mistakes on the rare class cost more during training. Oversampling duplicates or synthesises minority rows; undersampling removes majority rows. Weights change no data, oversampling can overfit, undersampling can throw away useful rows.
Explain it simply
Say a college quiz has 950 easy questions and 50 hard ones, and you want the student to take the hard ones seriously. You can (a) give hard questions 10 times the marks, (b) photocopy the hard ones until they match the easy ones, or (c) throw away most of the easy ones. Those are class weights, oversampling and undersampling.
Example
Scikit-learn's class_weight="balanced" uses weight = n_samples / (n_classes x count of that class).
1,000 rows: 950 class 0, 50 class 1
weight of class 0 = 1000 / (2 x 950) = 0.526
weight of class 1 = 1000 / (2 x 50) = 10.0An error on a class-1 row now costs about 19 times an error on a class-0 row (10 / 0.526).
from sklearn.linear_model import LogisticRegression
model = LogisticRegression(class_weight="balanced")Oversampling: random oversampling copies minority rows (risk: memorising them). Undersampling: random removal of majority rows (risk: losing information, fine when you have millions of rows).
Whichever you pick, apply it only to the training split, and judge the result with recall, precision or PR-AUC on an untouched test set.
What to say in the interview
"I start with class weights since they are simple and lose no data. If that is not enough I try oversampling or SMOTE, or undersampling when data is huge. I resample only the training data and evaluate on the original distribution. Often adjusting the decision threshold helps as much as resampling."
Common mistakes
- Resampling before the train-test split, so duplicates land in both sets.
- Expecting resampling to fix a lack of real minority examples.