How does the chi-square test of independence work?
It compares observed counts in a contingency table with the counts expected if the two categorical variables were independent. The statistic is the sum of (observed - expected)^2 / expected; a large value means the variables are related.
Explain it simply
You have two categories, say layout (old or new) and whether a user clicked. If the layout had no effect, the click share would be the same in both rows. Chi-square measures how far the real table is from that "no relationship" table.
Expected count for a cell = row total x column total / grand total.
Example
200 users saw either layout:
| Clicked | Did not click | Total | |
|---|---|---|---|
| Old | 30 | 70 | 100 |
| New | 45 | 55 | 100 |
Overall click rate is 75/200 = 37.5%, so expected clicks are 37.5 in each row and 62.5 non-clicks.
Chi-square = (30-37.5)^2/37.5 + (70-62.5)^2/62.5 + (45-37.5)^2/37.5 + (55-62.5)^2/62.5 = 1.5 + 0.9 + 1.5 + 0.9 = 4.8
With 1 degree of freedom the 5% cut-off is 3.84, so 4.8 is significant.
from scipy.stats import chi2_contingency
obs = [[30, 70], [45, 55]]
chi2, p, dof, exp = chi2_contingency(obs, correction=False)
print(round(chi2, 2), round(p, 4), dof) # 4.8 0.0285 1What to say in the interview
"Chi-square independence tests whether two categorical variables are associated, by comparing observed with expected counts. The degrees of freedom are (rows - 1) x (columns - 1). It needs counts rather than percentages, independent observations, and expected counts of roughly 5 or more per cell."
Common mistakes
- Running it on percentages instead of raw counts.
- Concluding it shows how strong the link is. It only tests whether there is one.