What is multicollinearity, and how do you detect it?
Multicollinearity is when predictors are strongly correlated with each other, which makes coefficient estimates unstable and hard to interpret. Detect it with a correlation matrix or variance inflation factor; VIF above about 5 to 10 is a warning.
Explain it simply
Suppose you predict a flat's price using area in square feet and area in square metres. The two columns carry the same information. The model cannot decide which one deserves the credit, so the coefficients become unstable: small changes in data swing them wildly, and they might even get opposite signs. Predictions are usually still fine; the explanation is what breaks.
Example
VIF for a feature = 1 / (1 - R2), where R2 comes from regressing that feature on all the other features.
- If the other features explain 90% of a feature's variation: VIF = 1 / (1 - 0.9) = 10
- If they explain 80%: VIF = 1 / 0.2 = 5
for r2 in (0.8, 0.9):
print(r2, round(1 / (1 - r2), 1)) # 0.8 5.0 / 0.9 10.0Ways to deal with it:
- Drop one of the near-duplicate features.
- Combine them, for example into a ratio or an average.
- Use regularisation such as Ridge, or PCA.
What to say in the interview
"Multicollinearity means predictors overlap heavily, inflating the variance of coefficients. I check pairwise correlations and the variance inflation factor, treating VIF above 5 to 10 as a concern. If the aim is prediction I may leave it; if the aim is interpretation I remove or combine features or use Ridge."
Common mistakes
- Checking only pairwise correlation. A feature can be a combination of several others without any single high pair.