How do you find and handle missing values in pandas?
Use isna().sum() to count them, then dropna to remove rows or columns, or fillna to fill them with a constant, mean, median or group value. Check what share is missing first, since the right fix depends on it.
Explain it simply
In pandas a gap shows up as NaN. First measure the problem, then decide whether to delete the rows, fill the gaps or leave them for a model that can handle them.
Example
import pandas as pd
d = pd.DataFrame({"a": [1, None, 3], "b": [None, None, 6]})
print(d.isna().sum().to_dict()) # {'a': 1, 'b': 2}
print(d.isna().mean().round(2).to_dict()) # {'a': 0.33, 'b': 0.67}
print(d.dropna().shape) # (1, 2)
print(d.dropna(how="all").shape) # (2, 2)
print(d["a"].fillna(d["a"].mean()).tolist()) # [1.0, 2.0, 3.0]Column b is two-thirds missing. dropna() with defaults drops any row with a NaN, leaving 1 row of 3. how="all" only drops rows where everything is missing, leaving 2.
Quick guide:
- A tiny share of rows missing at random: dropping is usually fine.
- Skewed numeric column: fill with the median.
- Category: fill with "Unknown" or the mode.
- Mostly empty column: consider dropping it, or adding a flag column
col_missing.
What to say in the interview
"I start with isna().mean() to see the percentage missing per column and ask why it is missing. Then I drop, impute with median or mode, or group-wise values, and sometimes add a missing indicator. I fit imputation on training data only, to avoid leakage."
Common mistakes
- Filling with the mean before splitting into train and test, leaking test information.