Interview Q&A / Data Science

1. How do you decide whether an A/B test result on conversion rate is significant?

   Compare the two conversion rates with a two-proportion z-test: z is the difference in rates divided by the pooled standard error. A p-value below your preset alpha, with a practically useful lift, supports shipping the change.

   IntermediateQuick answer1 min read
2. How does a two-sample t-test work? Walk through a small example.

   It tests whether two group means differ by more than chance would explain. The t statistic is the difference in means divided by its standard error; a large |t| gives a small p-value and evidence against equal means.

   IntermediateQuick answer1 min read
3. How does the chi-square test of independence work?

   It compares observed counts in a contingency table with the counts expected if the two categorical variables were independent. The statistic is the sum of (observed - expected)^2 / expected; a large value means the variables are related.

   IntermediateQuick answer1 min read
4. What is the difference between a one-tailed and a two-tailed test?

   A two-tailed test looks for a difference in either direction; a one-tailed test looks only for an increase or only for a decrease. The one-tailed p-value is half the two-tailed one, so the direction must be decided before seeing data.

   IntermediateQuick answer1 min read
5. When do you use a paired t-test instead of an independent t-test?

   Use a paired t-test when the two measurements come from the same subjects or matched pairs, such as before and after. It tests whether the mean of the differences is zero, which removes person-to-person variation.

   IntermediateQuick answer1 min read