Business continuity
Backups, retention, recovery objectives, and the procedure for each disaster scenario.
Four independent copies of different things
| Asset | Backed up | Where | Retention |
|---|---|---|---|
| The analytical database — events, entities, observations, graph, API key digests | Continuously, with daily automated snapshots | Encrypted, stored separately from the primary | 7 days standard, 28 days on enterprise agreements |
| Bulk extracts and raw source archives | Written to object storage as produced | Cloudflare R2 — a different provider from the database | Rolling |
| Schema and infrastructure definition | Every change, as a versioned migration in source control | GitHub, and every engineer's local clone | Permanent, full history |
| Application code and configuration | Every commit; each deployment is an immutable version | GitHub and the platform's version history | Permanent |
Backups are encrypted at rest and held separately from the primary, so a compromise of the running database is not a compromise of its history.
The record can also be rebuilt from source material: everything in the database derives from public feeds we can fetch again, and the raw archives sit with a different provider. Total loss of the database would be slow and expensive to recover from, but it would not be permanent loss of the product.
Committed objectives, by tier
| Objective | Standard | Enterprise |
|---|---|---|
| RPO — maximum data loss | 24 hours | 5 minutes |
| RTO — time to restore service | 4 hours | 2 hours |
| Backup retention | 7 days | 28 days |
| Recovery granularity | Daily snapshot | Any point in the retention window |
Point-in-time recovery, which is what brings the recovery point objective to minutes, is enabled for enterprise agreements. These are the objectives we commit to contractually.
The edge does not require recovery in the same sense. The API, Terminal and Atropos are stateless and globally distributed: a failed location is routed around automatically, and a faulty deployment is recovered by redeploying the previous immutable version, which takes about a minute.
Five scenarios and the response to each
| Scenario | Impact | Response | Expected |
|---|---|---|---|
| Faulty deployment | API errors or incorrect output | Redeploy the previous immutable version. No restore, no data involved. | Under 5 minutes |
| Edge location or region failure | None visible | The platform routes around it. No action required. | Automatic |
| Database unavailable | Cached reads continue; fresh reads and writes fail | Platform failover, or restore to a new instance and repoint the connection layer | Within RTO |
| Database corrupted or destroyed | Loss of the durable record | Restore to the recovery point, repoint, verify with the automated integrity checks, then re-ingest any gap from the public sources | Within RTO; any gap backfills behind it |
| Provider account compromised | Potentially total | Incident response: revoke every credential, rotate every secret, restore from backup into a clean project, rebuild the edge from source control. Everything required is in the repository. | Same day to service |
The published summary of the runbook
The full runbook is operational and is provided under NDA. Its structure:
- Declare. Classify the incident and start the notification deadlines in incident response. Nothing waits for the technical work to finish.
- Contain. Stop the harm before diagnosing it — disable the affected path, roll back, or revoke the credential.
- Restore. Recover the database to a new instance from the chosen recovery point, repoint the connection layer, and redeploy the edge from the last known good version.
- Verify before declaring recovery. Run the automated integrity checks against the restored system and read the output. A restored database that has lost an index, a trigger or a licence filter looks healthy until it is queried.
- Backfill the window between the recovery point and the failure from the public sources.
- Report. A written post-incident report within 5 business days.
What is tested and how often
- Rollback is exercised continuously, because it is the same mechanism as an ordinary deployment rather than a separate procedure.
- Restore from backup is tested to a scratch instance and the result written up: what was restored, from which backup, how long it took, and what failed. That report forms part of the evidence pack. Testing that a backup restores is a different thing from testing that it exists.
- The automated integrity checks are the verification step, which is what allows a restore to be trusted: they establish that the recovered system is correct rather than merely running.
- A full disaster-recovery exercise is scheduled annually, against a written test plan, with the report shared with enterprise customers on request.
The key-person question
Vaethra is a small company, and the material continuity risk is people rather than infrastructure.
- Nothing depends on anyone's memory. Schema, configuration, site content and deployment are all in source control, and the repository carries the reasoning behind each decision as well as the change itself. That documentation is a continuity control.
- Credential recovery does not depend on one person being reachable. Account recovery paths and offline recovery codes are held so that the estate can be reached in a key-person event.
- Enterprise agreements can include source-code escrow, releasing on defined insolvency or cessation events.
- The data is reproducible. Every record derives from a public source that still exists.