Connection pool exhaustion: symptoms and diagnosis
The signature of an exhausted connection pool is an API that is fast for one request and falls apart at a specific concurrency level: below the pool size, requests grab a connection instantly; above it, they queue waiting for one to free up. The result is a bimodal response-time distribution — a fast cohort and a slow cohort — rather than uniform slowness. That two-cluster shape is how you tell pool exhaustion apart from “the query is just slow.”
The symptoms, in order of appearance
- Single requests are fine. curl it alone: 40ms. Every synthetic health check passes.
- p95 cliffs at a threshold. At 10 concurrent users everything’s normal; at 25, p95 jumps 10× while p50 barely moves — the lucky half still gets connections instantly.
- Latency splits into two clusters. Some requests finish in ~40ms (got a connection), the rest in ~500ms+ (queued). Averages land uselessly in between, on a value almost no request actually experienced.
- Timeouts at peak. Push further and queued requests exceed the pool’s acquire timeout —
connection pool timeout,too many clients already(Postgres), or plain 500s. - Instant recovery when load drops. No lingering badness — the moment concurrency dips below the pool size, everything is fast again. Memory leaks and cache stampedes don’t recover like that; pool queues do.
Why it happens
Every framework’s database client keeps a fixed pool of open connections — often a default of 5 or 10. Each in-flight request that touches the database holds one connection for the duration of its queries. When concurrent requests exceed pool size × (1 / query-time), arrivals must wait in the pool’s queue. The wait time is invisible in your database’s own metrics — the database sees only the queries it’s running, all of them fast. The time is lost in your app, waiting in line, and it lands in your endpoint’s TTFB.
How to confirm it
Provoke it deliberately: hold N requests in flight against the endpoint and sweep N upward. If the TTFB distribution splits into two cohorts right around your configured pool size, you’ve found it. Lobe’s concurrency probe automates exactly this sweep and flags the split:
lobe probe https://api.example.com/users --concurrency 1,5,10,25,50
── ttfb vs concurrency ──
conc p50 p95
1 42ms 47ms
5 43ms 52ms
25 46ms 607ms ← bimodal
bimodal ttfb: fast cohort 43ms ×81 (54%) / slow cohort 604ms ×69 (46%)The cliff sits between 5 and 25 — and the cohort split at 25 says “bounded resource,” not “slow code.”
Fixes, in the order to try them
- Make queries faster — pool capacity is pool size ÷ query time, so halving a query’s duration doubles effective capacity for free.
- Release connections sooner — don’t hold one across an external API call or slow rendering.
- Size the pool to reality — match it to expected concurrency, bounded by your database’s connection limit (and use a server-side pooler like PgBouncer when many app instances share one database).