SEV-2 · RESOLVED

API latency spike — 2026-05-22

Elevated p99 on the deploy endpoint for 41 minutes.

Impact
~3% of writes > 5s
Duration
41 min
Root cause
Connection pool exhaustion

Timeline (UTC)

14:02
Deploy of v2.9 ships a missing pool-size cap.
14:09
p99 crosses 5s; alert fires.
14:31
On-call identifies pool saturation, scales connections.
14:43
Latency back to baseline; incident resolved.

Action items

① Cap pool size in config (done). ② Add saturation alert at 70%. ③ Load-test the deploy path in CI.