When should you use median instead of mean to fill missing values?
Use the median when the column has outliers or is skewed, because the mean gets dragged by extreme values. Use the mean for roughly symmetric data without outliers.
Explain it simply
Think of the monthly salaries in a small startup: four employees earn about 30,000 rupees and the founder takes 500,000. If you fill a missing salary with the "average", you will invent a salary no ordinary employee earns. The median, the middle value, is not pulled by the founder's pay.
Example
Salaries (thousands): 30, 32, 35, 31, 500
Mean = (30+32+35+31+500) / 5 = 628 / 5 = 125.6
Median = sorted 30, 31, 32, 35, 500 -> middle = 32Filling a missing salary with the mean gives 125.6, higher than four of the five real values. The median gives 32, which looks like a typical employee.
import numpy as np
x = np.array([30, 32, 35, 31, 500])
print(x.mean(), np.median(x)) # 125.6 32.0For categorical columns use the most frequent value (mode) or a separate "Unknown" category. For symmetric data such as heights, mean and median are close and either is fine.
What to say in the interview
"The mean is sensitive to outliers and skew, so for income-like or heavy-tailed columns I use the median. For symmetric data the mean is fine. For categories I use the mode or an explicit Unknown category. Either way I compute it on training data only."
Common mistakes
- Using the mean on a column with obvious outliers.
- Using the median on a categorical column; it has no meaning there.