Why do engineers track p95 and p99 latency instead of the average?
Averages hide slow outliers. p99 means 99% of requests were faster than this value, so it shows what your unluckiest users feel, which is what drives complaints and timeouts.
Explain it simply
Imagine a college canteen where 99 students get tea in 1 minute and one student waits 30 minutes. The average is about 1.3 minutes, which sounds fine, yet one person had a terrible experience. The 99th percentile (p99) would show 30 minutes.
A percentile answers: "what fraction of requests finished within this time?" p50 is the median, p95 means 95% were at least that fast, p99 covers all but the slowest 1%.
This matters more than it looks. A single page may call ten services; if each has a slow 1% tail, roughly one page load in ten hits at least one slow call. Heavy users, who make the most requests, meet the tail most often.
Example
Ten request times in milliseconds:
20, 22, 21, 23, 20, 22, 24, 21, 22, 900
Average = 1074 / 10 = 107.4 ms (looks acceptable)
Median = 22 ms
Max = 900 ms (one user waited almost a second)The average describes nobody: most users saw about 22 ms and one saw 900 ms.
What to say in the interview
"The average hides the tail. I report p50 for the typical case and p95 or p99 for the worst experiences, and I set SLOs on those percentiles. Tail latency compounds across fan-out calls, so p99 of each downstream service matters to the whole request."
Common mistakes
- Averaging percentiles across servers; merge the raw histograms instead.