What is the difference between linear regression and logistic regression?
Linear regression predicts a continuous number, like a house price, by fitting a straight line. Logistic regression predicts the probability of a class, like spam or not spam, by passing a linear score through the sigmoid function so the output stays between 0 and 1.
Explain it simply
Both models start the same way: they multiply each input by a weight, add them up, and add a constant. The difference is what they do with that total.
- Linear regression uses the total directly as the answer. "Given hours studied, the predicted marks are 62." The answer can be any number.
- Logistic regression squashes the total into a probability between 0 and 1. "Given this applicant's income and credit history, the chance of repaying the loan is 0.83." You then pick a threshold, often 0.5, to say yes or no.
Despite the name, logistic regression is used for classification.
Example
Linear regression: predicting marks
Say the model learnt: marks = 30 + 8 × hours.
- 4 hours → 30 + 32 = 62 marks
- 10 hours → 30 + 80 = 110 marks
The second answer is above 100, which shows a limitation: a straight line keeps going and does not know the valid range.
Logistic regression: predicting pass or fail
Say the model learnt: z = −4 + 1 × hours, and probability of passing = sigmoid(z) = 1 / (1 + e^(−z)).
- 4 hours → z = 0 → probability = 0.5
- 6 hours → z = 2 → probability ≈ 0.88
- 2 hours → z = −2 → probability ≈ 0.12
However large z gets, the probability never goes above 1 or below 0.
How it works
| Linear regression | Logistic regression | |
|---|---|---|
| Task | Regression (numeric output) | Classification (probability of a class) |
| Output | Any real number | Between 0 and 1 |
| Function | ŷ = w·x + b | p = sigmoid(w·x + b) |
| Loss | Mean squared error | Log loss (binary cross-entropy) |
| Typical metrics | MAE, RMSE, R² | Accuracy, precision, recall, F1, ROC-AUC |
Linear regression with mean squared error has a closed-form solution (the normal equation), and it can also be trained with gradient descent. Logistic regression has no closed-form solution, so it is trained with iterative methods like gradient descent. Log loss is used instead of squared error because it heavily punishes confident wrong answers and gives a convex problem that is easy to optimise.
In logistic regression, w·x + b is the log-odds, log(p / (1 − p)). So each weight tells you how much one unit of that feature changes the log-odds of the positive class. The decision boundary, where p = 0.5, is a straight line (or flat plane) in the input space, which is why logistic regression is called a linear classifier.
Real-world use
- Linear regression: predicting house rent from area and location, estimating crop yield from rainfall and fertiliser, forecasting sales.
- Logistic regression: loan approval, spam detection, predicting whether a customer will cancel a subscription. Banks like it because each weight is easy to explain to a regulator.
Common mistakes
- Saying logistic regression is a regression algorithm for numbers. It predicts class probabilities.
- Using linear regression for a yes/no target. Its outputs can go below 0 or above 1 and do not behave like probabilities.
- Forgetting to scale features when using regularisation or gradient descent.
- Assuming the threshold must be 0.5. Choose it based on the cost of false positives versus false negatives.
What to say in the interview
"Both compute a weighted sum of the inputs. Linear regression outputs that sum directly, so it predicts continuous values like price, and is usually trained with mean squared error. Logistic regression passes the sum through the sigmoid function to get a probability between 0 and 1, and is trained with log loss, so it is used for classification like spam detection or loan approval. Its weights are interpretable as changes in log-odds, and its decision boundary is linear. For more than two classes we use softmax, called multinomial logistic regression."
Follow-up questions
- Why do we use log loss instead of mean squared error for logistic regression?
- What assumptions does linear regression make?
- How do you extend logistic regression to more than two classes?
- What does R² measure?
Go deeper
Read these once the basics above feel easy.
Classic linear regression assumes a linear relationship, independent errors, constant error variance (homoscedasticity) and, for valid confidence intervals, roughly normal errors. Strong correlation between features (multicollinearity) makes individual weights unstable even if predictions are fine; L2 regularisation (ridge) helps. Logistic regression can also be regularised, and in scikit-learn LogisticRegression applies L2 regularisation by default, controlled by C, where a smaller C means stronger regularisation.
#regression#classification#logistic-regression#linear-regression