500 Errors on Admin and Storefront

Incident Report for Uscreen

Postmortem

Summary

On September 6, 2025, Uscreen experienced a service outage that affected access to the Admin Area, storefronts, and API. Our engineering team identified the root cause, implemented a workaround, and fully restored services within approximately 1 hour and 45 minutes.

Root Cause

The outage was triggered by an issue with one of our third-party service providers responsible for email delivery. A network failure on the service provider's end prevented our platform from connecting to the provider, which in turn caused issues with our own infrastructure, preventing Uscreen from functioning properly.

Resolution

Our engineering team isolated the dependency and deployed a temporary fix that allowed the platform to operate independently of the impacted service provider. This restored storefronts, the Admin Area, and API functionality while email delivery remained degraded.

Once the provider resolved their issue, email services returned to normal and all systems were fully restored.

Preventative Measures

To reduce the likelihood of similar incidents in the future, we are making the following improvements:

  • Adjusting our system architecture so external services cannot cause full outages.
  • Updating health checks to ensure that Uscreen remains available even if non-critical services are unavailable.
  • Introducing redundancy and fallback mechanisms for critical email communications delivery.
  • Reviewing other external service dependencies to ensure resilience.

Moving Forward

We know how disruptive incidents like this can be and we sincerely apologize for the impact it had. While every online platform relies on certain third-party providers, our focus is on making sure that interruptions from those services have limited impact for Uscreen.

Thank you for your continued trust and for being part of the Uscreen community.

Posted Sep 08, 2025 - 15:06 UTC

Resolved

The issue impacting access to the Admin Area and storefronts has been fully resolved. Our team has confirmed that services are stable and no further issues have been identified.

Thank you for your patience and understanding while we worked to restore service.
Posted Sep 06, 2025 - 17:57 UTC

Monitoring

Services in the Admin Area and storefronts are now back online. We are continuing to monitor the platform closely to ensure stability and confirm that the issue is fully resolved.
Posted Sep 06, 2025 - 17:06 UTC

Identified

Our team has identified the issue with our cloud service partner that was causing errors in the Admin Area and storefronts. We are actively working on restoring the platform and are beginning to see improvements. We will continue to monitor progress closely and provide further updates as recovery continues.
Posted Sep 06, 2025 - 17:02 UTC

Update

Our team continues to actively work with our cloud service partner to investigate the issue impacting the Admin Area and storefronts. We are monitoring the situation closely and remain focused on restoring normal service as quickly as possible.

Further updates will be shared as soon as more details are available.

Thank you for your continued patience.
Posted Sep 06, 2025 - 16:57 UTC

Update

Our team continues to investigate an issue with one of our cloud service partners that may cause error messages in both the Admin Area and storefronts. Our engineering team is actively working with the provider to identify the cause and restore normal service as quickly as possible. We will share updates as soon as we have more information.
Posted Sep 06, 2025 - 16:23 UTC

Investigating

We are currently experiencing an issue that may affect some of our services. Our team is actively investigating and working to resolve it as quickly as possible. Thank you for your patience.
Posted Sep 06, 2025 - 16:12 UTC
This incident affected: Admin Portal and Storefront.