Postmortem -
Read details
May 4, 17:32 UTC
Resolved -
This incident is now resolved. All services are operating normally. A detailed post-mortem will be shared shortly. Thank you for your patience.
May 4, 16:46 UTC
Update -
All services are back online and operating normally. We're continuing to monitor closely to ensure everything remains stable. We'll share a final update once we're confident the issue is fully resolved.
May 4, 15:31 UTC
Monitoring -
Mobile and TV app APIs are coming back online. We're continuing to monitor as services fully recover. We'll share another update once everything is operating normally.
May 4, 15:22 UTC
Update -
Storefronts and the Admin Portal are now fully back online and running normally. Thanks for your patience while we worked through this. The APIs that power mobile and TV apps are still affected — that's our team's full focus right now. We'll update again as soon as those are restored.
May 4, 14:58 UTC
Update -
Good news: storefronts and the Admin Portal are recovering. We're now focused on bringing the APIs that power mobile and TV apps back online. We'll update again as soon as more is restored.
May 4, 14:47 UTC
Update -
Recovery is underway and performance is improving across our services. We're proceeding carefully to ensure stability as everything returns to normal. You may still see some degradation for a short while longer. We'll update as soon as full recovery is confirmed.
May 4, 14:30 UTC
Update -
The earlier fix did not hold and we're seeing degradation return. Engineering is investigating further. We'll update as soon as we have more.
May 4, 13:42 UTC
Update -
We've rolled out a fix and performance is returning to normal, while we improving the cache ratio. We're continuing to monitor closely to make sure everything stays stable. We'll share a final update once we're confident the issue is fully resolved.
May 4, 13:12 UTC
Identified -
We've identified the cause of the slow page loads and errors affecting the storefront, admin portal, and APIs. Following an earlier database upgrade, a subset of queries are running significantly slower than expected. Our team is implementing a fix now. We'll share another update as soon as the fix is deployed and we begin monitoring recovery.
May 4, 12:31 UTC
Update -
We are continuing to investigate this issue.
May 4, 12:06 UTC
Investigating -
We are currently investigating an issue causing degraded performance in Communities. We'll post updates as soon as they are available. Thank you for your patience.
May 4, 11:37 UTC