Why do we divide by n-1 when computing sample variance?
A sample sits closer to its own mean than to the true population mean, so dividing by n underestimates the population variance. Dividing by n-1 (Bessel correction) removes that bias on average.
Explain it simply
When you compute deviations, you measure them from the sample mean, and the sample mean is built from the same data points. So the points are always a bit closer to it than they would be to the unknown true mean. The spread looks smaller than it is.
Another way to see it: once the mean is fixed, only n-1 deviations are free. The last one is forced so that all deviations add up to zero. These are the degrees of freedom.
Example
Values 2, 3, 4, 5, 6 have mean 4. Squared deviations: 4, 1, 0, 1, 4 = 10.
- Divide by n: 10 / 5 = 2.0
- Divide by n-1: 10 / 4 = 2.5
import numpy as np
a = [2, 3, 4, 5, 6]
print(np.var(a)) # 2.0 (ddof=0, divides by n)
print(np.var(a, ddof=1)) # 2.5 (divides by n-1)Note that NumPy divides by n by default, while pandas .var() and .std() divide by n-1 by default.
What to say in the interview
"Sample variance divides by n-1 because the deviations are taken from the sample mean, which uses up one degree of freedom. This makes the estimator unbiased for the population variance. For a whole population, or for a very large sample, dividing by n is fine."
Common mistakes
- Forgetting the NumPy versus pandas default difference, and getting two different answers.
- Saying n-1 makes the standard deviation unbiased. It fixes the variance; the standard deviation stays slightly biased.