What is multicollinearity, and how do you detect and fix it?
Multicollinearity means two or more features are strongly correlated, so a linear model cannot tell their effects apart and coefficients become unstable. Detect it with a correlation matrix or VIF above about 5-10. Fix it by dropping or combining features, PCA, or ridge regression.
Explain it simply
Suppose a house-price model uses both "area in square feet" and "area in square metres". They carry the same information. The model can give 100 to one and 0 to the other, or 50 to each, and fit equally well. Small changes in data swing the coefficients wildly, even though predictions stay fine.
Example
Variance Inflation Factor for a feature = 1 / (1 - R^2), where R^2 comes from regressing that feature on all the other features.
If another-features-predict-this-one R^2 = 0.90 -> VIF = 1 / 0.10 = 10
If R^2 = 0.50 -> VIF = 2
If R^2 = 0 -> VIF = 1 (no collinearity)from statsmodels.stats.outliers_influence import variance_inflation_factor
vif = variance_inflation_factor(X.values, i) # for column iFixes:
- Drop one of the duplicated features.
- Combine them (for instance area per room).
- Apply PCA to get uncorrelated components.
- Use ridge regression (L2), which shrinks and stabilises coefficients.
Tree models are not much bothered by it for prediction, though feature importances get split between correlated features.
What to say in the interview
"Multicollinearity inflates coefficient variance and makes them hard to interpret, while predictions can remain accurate. I detect it with a correlation heatmap and VIF, treat above 5 or 10 as a warning, and fix it by removing or merging features, PCA or ridge."
Common mistakes
- Dropping features based on correlation with the target instead of with each other.
- Worrying about it when only prediction accuracy matters.