How would you encode a categorical feature with thousands of values, like a pincode?
One-hot would create thousands of sparse columns. Better options are target (mean) encoding computed out-of-fold, frequency encoding, grouping rare values into 'other', hashing, or learned embeddings.
Explain it simply
India has roughly 19,000 pincodes. One-hot encoding turns one column into 19,000, most of them nearly always zero. Models get slow, memory grows, and each pincode has only a handful of rows, so learning from it is noisy.
The idea is to keep the useful signal (does this area tend to have higher prices, or more defaults?) in one or a few columns.
Example
Options:
1. Frequency encoding : replace pincode by how often it appears
2. Target encoding : replace pincode by the average target for it
3. Group rare values : pincodes with < 20 rows become "other"
4. Coarsen : use first 3 digits (a postal region), or city
5. Hashing : hash into a fixed number of buckets
6. Embeddings : a neural network learns a short vector per codeTarget encoding example: pincode 560001 has 40 past loans, 6 defaulted, so its encoded value is 6/40 = 0.15.
The danger is leakage: the target is used to build the feature. Compute it out-of-fold, and smooth rare codes toward the global mean, for instance with TargetEncoder in scikit-learn (1.3 and later), which does cross-fitting internally.
What to say in the interview
"I avoid plain one-hot for high cardinality. I would first group rare values or coarsen the code, then use target encoding with cross-fitting and smoothing, or frequency encoding for tree models. The main risk is target leakage, so the encoding must be learned only on training folds."
Common mistakes
- Target encoding on the full training set and then validating on the same rows.
- Ignoring unseen pincodes at prediction time.