What is Laplace smoothing in Naive Bayes, and why is it needed?
If a word never appeared with a class in training, its probability is zero and wipes out the whole product. Laplace smoothing adds 1 to every word count so no probability is exactly zero.
Explain it simply
A judge who has never seen a particular excuse before declares "impossible, this person is innocent forever" because the probability is zero. Smoothing is the judge saying "I have not seen it yet, but I will not rule it out".
Example
Suppose spam emails in training contain 100 word tokens in total and the vocabulary has 50 distinct words. The word "scholarship" appears 0 times in spam.
Without smoothing: P(scholarship | spam) = 0 / 100 = 0
-> any email with "scholarship" gets spam score 0, whatever else it says
With Laplace (alpha = 1):
P(scholarship | spam) = (0 + 1) / (100 + 50) = 1/150 = 0.0067
A word seen 10 times:
P = (10 + 1) / (100 + 50) = 11/150 = 0.0733The denominator adds the vocabulary size so the probabilities still sum to 1.
from sklearn.naive_bayes import MultinomialNB
MultinomialNB(alpha=1.0) # alpha = 1 is Laplace smoothingSmaller alpha (such as 0.1) trusts the training counts more; larger alpha pushes probabilities toward uniform. Tune alpha by cross-validation.
What to say in the interview
"Without smoothing, one unseen word makes a class probability zero and cancels all other evidence. Laplace smoothing adds a pseudo-count of 1 to each word and the vocabulary size to the denominator. In scikit-learn it is the alpha parameter, which I tune."
Common mistakes
- Forgetting to add the vocabulary size to the denominator.
- Setting alpha to 0 and then seeing zero-probability errors.