How does logistic regression turn a score into a probability?
It computes a linear score z = w.x + b and passes it through the sigmoid 1/(1+e^-z), which squashes any number into 0 to 1. A threshold, usually 0.5, then converts the probability into a class.
Explain it simply
A linear score can be any number: -40, 3, 900. A probability must sit between 0 and 1. The sigmoid is the squashing function that converts one into the other: very negative scores become close to 0, very positive ones close to 1, and a score of 0 becomes exactly 0.5.
Example
Model for "will the student pass?" with z = 0.5 x hours - 3.
hours = 2 -> z = -2 -> p = 1/(1+e^2) = 1/(1+7.389) = 0.119
hours = 6 -> z = 0 -> p = 1/(1+1) = 0.500
hours = 8 -> z = 1 -> p = 1/(1+e^-1) = 1/(1+0.368) = 0.731At a 0.5 threshold, 2 hours predicts fail, 8 hours predicts pass. Studying 6 hours sits exactly on the decision boundary, where z = 0.
import numpy as np
def sigmoid(z):
return 1 / (1 + np.exp(-z))
print(round(sigmoid(1), 3)) # 0.731The decision boundary w.x + b = 0 is a straight line (or plane), which is why logistic regression is a linear classifier.
What to say in the interview
"Logistic regression computes a linear score and applies the sigmoid to get a probability between 0 and 1. I then apply a threshold, which I can move to trade precision against recall. The boundary is linear, so it cannot separate classes that need a curved boundary unless I add features."
Common mistakes
- Saying the output is a class label; predict_proba gives probabilities and predict applies the threshold.
- Forgetting that the threshold is adjustable.