AI Provider Outage Playbook

Provider instability is not only a backend concern

Many teams initially treat provider outages as occasional timeout spikes. In production, the blast radius is wider:

• success rate drops across user-facing flows
• latency stretches long enough for users to abandon
• retries amplify cost at the worst possible moment
• support and operations hear about the problem before engineering fully sees it

That is why provider outages need an explicit operating playbook.

Define what counts as an incident before one starts

The most common failure is not missing alerts. It is lacking a shared threshold for action.

Set at least three classes of triggers:

• error-rate thresholds over a defined time window
• latency thresholds such as a sustained P95 increase
• capability thresholds for workflows like image generation, long context, or structured output

Without explicit triggers, teams stay in a vague “watching” mode for too long.

Write the degradation path as an ordered sequence

Incident response breaks down when everyone has a different fallback idea in mind.

A safer degradation chain usually looks like:

1. switch to a steadier model within the same provider
2. if instability continues, move traffic to a second provider
3. temporarily tighten quotas on expensive or long-running flows
4. disable non-critical entry points to protect the core path

The point is not theoretical optimality. It is repeatable execution under pressure.

Communication should start before the root cause is final

Teams often wait too long to communicate because they want full certainty first.

In practice, split communication into two channels:

• internal updates for engineering, support, and operations
• user-facing notices that explain which workflows are degraded and what action users should take

The goal is expectation management, not a perfect technical explanation.

Postmortems should ask why the switch did not happen sooner

Once the incident is over, avoid stopping at “the upstream provider was unstable.”

More useful questions are:

• were alerts early enough to matter
• did fallback require too much manual judgment
• did degradation push cost beyond acceptable limits
• could users still complete the critical workflow

A good outage playbook turns provider instability into a controlled operating event instead of a recurring surprise.