What is the difference between normalisation and standardisation?
Normalisation (min-max scaling) squeezes values into a fixed range, usually 0 to 1, using (x - min) / (max - min). Standardisation (z-score scaling) shifts values to mean 0 and standard deviation 1, using (x - mean) / std. Standardisation handles outliers better, and both should be fitted on training data only.
Explain it simply
Suppose a model looks at two features of a job applicant: years of experience (0 to 10) and salary in rupees (3,00,000 to 30,00,000). To many algorithms, salary looks a lakh times more important simply because its numbers are bigger. Scaling puts features on a comparable footing.
- Normalisation is like converting every exam to a score out of 1. The lowest becomes 0, the highest becomes 1.
- Standardisation is like saying how far each student is above or below the class average, measured in standard deviations.
Example
Salaries of four employees, in lakh per year: 3, 5, 8, 12
Normalisation (min-max): min = 3, max = 12, range = 9.
- 3 → (3 - 3) / 9 = 0.00
- 5 → 2 / 9 ≈ 0.22
- 8 → 5 / 9 ≈ 0.56
- 12 → 9 / 9 = 1.00
Standardisation (z-score): mean = 7, standard deviation (dividing by n) ≈ 3.39.
- 3 → (3 - 7) / 3.39 ≈ -1.18
- 5 → -2 / 3.39 ≈ -0.59
- 8 → 1 / 3.39 ≈ 0.29
- 12 → 5 / 3.39 ≈ 1.47
import numpy as np
from sklearn.preprocessing import MinMaxScaler, StandardScaler
X = np.array([[3], [5], [8], [12]])
print(MinMaxScaler().fit_transform(X).round(2).ravel()) # [0. 0.22 0.56 1. ]
print(StandardScaler().fit_transform(X).round(2).ravel()) # [-1.18 -0.59 0.29 1.47]How it works
| Normalisation (min-max) | Standardisation (z-score) | |
|---|---|---|
| Formula | (x - min) / (max - min) | (x - mean) / std |
| Output range | Fixed, usually [0, 1] | Not bounded; mean 0, std 1 |
| Effect of one huge outlier | Squashes all other values close to 0 | Affects mean and std, but less badly |
| Good for | Pixel values, bounded inputs, some neural nets | Linear models, SVM, PCA, KNN, logistic regression |
Neither method changes the shape of the distribution. Skewed data stays skewed after scaling; to reduce skew you need a transform such as a log.
Real-world use
- Distance-based models (KNN, k-means) and SVMs need scaling, otherwise the largest-numbered feature dominates the distance.
- Gradient descent converges faster when features are on similar scales.
- PCA finds directions of maximum variance, so an unscaled large feature would dominate the components.
- Images: pixel values 0-255 are usually divided by 255, which is min-max scaling.
- Tree-based models (decision trees, random forest, gradient boosting) split on thresholds and generally do not need scaling.
Common mistakes
- Fitting the scaler on the full dataset before splitting. The test set's min, max or mean leaks into training. Fit on the training set, then
transformthe test set. - Using min-max scaling on data with extreme outliers. One ₹5 crore salary would push every normal salary close to 0.
- Assuming standardisation makes data normally distributed. It only shifts and rescales.
- Mixing up the names. Some people say "normalisation" for any scaling, so state the formula when you answer.
What to say in the interview
"Normalisation, or min-max scaling, maps each feature to a fixed range like 0 to 1 using x minus min over max minus min. Standardisation subtracts the mean and divides by the standard deviation, so each feature has mean 0 and standard deviation 1. I use standardisation by default for linear models, SVM, PCA and KNN because it is less distorted by outliers, and min-max when I need bounded values, such as image pixels. Tree models usually do not need scaling. In all cases I fit the scaler on the training data only."
Follow-up questions
- Which algorithms need feature scaling and which do not?
- Why must the scaler be fitted only on the training set?
- What is RobustScaler and when would you use it?
- Does standardisation make the data normally distributed?
Go deeper
Read these once the basics above feel easy.
RobustScaler in scikit-learn subtracts the median and divides by the IQR, so a few extreme values barely affect the scaling. It is a good choice when outliers are genuine and you do not want to remove them.
Inside a scikit-learn Pipeline, the scaler is refitted on each training fold during cross-validation, which prevents leakage automatically. Remember to save the fitted scaler along with the model, because new data at prediction time must be scaled with the same mean and standard deviation learnt during training.
#machine-learning#feature-scaling#preprocessing#scikit-learn