Interview Q&A / Data Science

1. What is hypothesis testing, and what does a p-value actually mean?

   Hypothesis testing checks whether data is surprising under a default assumption called the null hypothesis. The p-value is the probability of seeing results at least as extreme as yours if the null hypothesis were true. A small p-value is evidence against the null. It is not the probability that the null is true.

   AdvancedIn-depth5 min read
2. What are Type I and Type II errors in hypothesis testing?

   A Type I error is a false positive: rejecting a true null hypothesis, with probability alpha. A Type II error is a false negative: failing to reject a false null, with probability beta. Power is 1 - beta.

   BeginnerQuick answer1 min read
3. How do you decide whether an A/B test result on conversion rate is significant?

   Compare the two conversion rates with a two-proportion z-test: z is the difference in rates divided by the pooled standard error. A p-value below your preset alpha, with a practically useful lift, supports shipping the change.

   IntermediateQuick answer1 min read
4. How does a two-sample t-test work? Walk through a small example.

   It tests whether two group means differ by more than chance would explain. The t statistic is the difference in means divided by its standard error; a large |t| gives a small p-value and evidence against equal means.

   IntermediateQuick answer1 min read
5. How does the chi-square test of independence work?

   It compares observed counts in a contingency table with the counts expected if the two categorical variables were independent. The statistic is the sum of (observed - expected)^2 / expected; a large value means the variables are related.

   IntermediateQuick answer1 min read
6. What is the difference between a one-tailed and a two-tailed test?

   A two-tailed test looks for a difference in either direction; a one-tailed test looks only for an increase or only for a decrease. The one-tailed p-value is half the two-tailed one, so the direction must be decided before seeing data.

   IntermediateQuick answer1 min read
7. When do you use a paired t-test instead of an independent t-test?

   Use a paired t-test when the two measurements come from the same subjects or matched pairs, such as before and after. It tests whether the mean of the differences is zero, which removes person-to-person variation.

   IntermediateQuick answer1 min read