# Runbook — Primary Database Failover

**Service** core-db · **Severity when used** SEV-1 · **Owner** Platform

## When to use this

Primary is unreachable for >2 minutes AND automated failover
has not fired. If automation is mid-failover, **wait** — see
step 0.

## Procedure

0. Check automation state first:

```bash
nwctl failover status core-db
```

1. Confirm the primary is actually down (not a monitor flap):

```bash
nwctl db ping core-db --from eu-1,us-1
```

2. Promote the replica:

```bash
nwctl db promote core-db-replica-2 --confirm
```

3. Verify writes on the new primary before announcing:

```bash
nwctl db verify core-db --writes
```

## Decision points

| Situation                     | Action                     |
| ----------------------------- | -------------------------- |
| Replica lag > 30s at promote  | Page DBA on-call, hold     |
| Both replicas unhealthy       | Invoke DR plan, do NOT promote |
| Promote fails twice           | Escalate to vendor support |

## Aftercare

- Announce in #incidents with new primary id
- File the timeline within 24h — template in the postmortem repo