Previous incidents

We have identified and resolved the issue affecting backups and restorations. As a precaution, we have temporarily increased backup retention for all customers to safeguard against any corrupted backups created during the incident. Older backup restorations, manual backups, and scheduled backups are now fully operational. Thank you for your patience.

We have identified the root cause and have taken steps to remediate it. Recovery is now visible with performance returning to normal.

We have released the fix.

Closing the incident as services continue to perform realiably.

We’ve noticed a burst of errors in the NYC and SYC regions due to cache pollution. We’ve cleared the databases cache in affected regions, and all services are back online. Our engineering team is investigating to understand the root cause of the issue.

We've improved builds so they should process more reliably now. If you have any old deployments waiting, please trigger a new deployment.

The issue was quickly mitigated, but problem persisted in New York region. This region is now also stable, and we are seeing full recovery. We continue monitoring to ensure stability.

We've also made some improvements with DNS to help with random fetch failed errors in Sites.

We'll close this incident now, but continue to monitor for issues.