How can scaling or imputing before the train-test split cause data leakage?
If you compute the mean or maximum on the full dataset, information from test rows leaks into training. Always fit scalers and imputers on the training set only, then apply them to the test set. A scikit-learn Pipeline does this automatically.
Explain it simply
Imagine a teacher who sets the exam, then lets one student see the class average of the exam answers before writing it. The student's score now depends partly on information from the test itself. That is leakage.
If you standardise using the mean of all 1,000 rows and then split, the training rows have been shaped by the test rows' values. Your test score looks slightly better than reality.
Example
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
# Wrong: scaler = StandardScaler().fit(X) # sees the test rows
scaler = StandardScaler().fit(X_train) # learns mean/std from train only
X_train_s = scaler.transform(X_train)
X_test_s = scaler.transform(X_test) # reuse train mean/stdThe same rule applies to imputers, encoders, PCA and feature selection. Inside cross-validation it matters even more, because each fold needs its own fit. Putting the steps into Pipeline([("scale", StandardScaler()), ("model", LogisticRegression())]) and passing that to cross_val_score refits every step on each training fold.
What to say in the interview
"Anything that learns a statistic from data must be fitted on training data only: scaler, imputer, encoder, PCA, feature selector. I then transform the test data with those fitted values. A Pipeline enforces this, including inside cross-validation."
Common mistakes
- Calling fit_transform on the test set.
- Doing feature selection on all data and then cross-validating.