What does R-squared tell you, and what does it not tell you?
R-squared is the share of the variation in the target that the model explains, from 0 to 1. It says nothing about whether the model is correct, unbiased or good at predicting new data, and it never decreases when you add features.
Explain it simply
Marks vary from student to student. R-squared asks: how much of that variation does knowing the study hours account for? If R-squared is 0.9, 90% of the ups and downs in marks are explained by the model and 10% is left over.
R-squared = 1 - (squared errors of the model) / (squared errors of just predicting the mean)
Example
Using the study-hours data (x = 1 to 5, y = 40, 50, 55, 65, 70) and the line marks = 33.5 + 7.5 x hours:
- Total variation around the mean (56) = 570
- Squared errors left after the line = 7.5
- R-squared = 1 - 7.5 / 570 = 0.987
ss_res, ss_tot = 7.5, 570
print(round(1 - ss_res / ss_tot, 3)) # 0.987Adding useless features nudges R-squared up anyway. Adjusted R-squared penalises that: with n = 20 rows, R-squared 0.80 using 3 features gives adjusted 0.7625, while 0.82 with 10 features gives only 0.62.
What to say in the interview
"R-squared is the fraction of target variance explained by the model. A high value does not prove the model is right, because a curved relationship or overfitting can hide behind it, and a low value can still be useful in noisy fields. I use adjusted R-squared for comparing models with different numbers of features and check validation error."
Common mistakes
- Judging models by training R-squared alone.