Wait times are intermittently spiking to 100 or more seconds on all clouds. These spikes only last about three minutes and occur about once every two hours. We are investigating the cause.
2020-February-22 Service Incident
Incident Report for Sauce Labs US West Data Center


Saturday February 22 10:03 - 13:58 PT

What happened:

Saucelabs.com, app.saucelabs.com, UI, REST API, Sauce Connect tunnels and Automated Tests in our San Jose, CA datacenter were all offline.

Why it happened:

An internal service that secures communications inside our Kubernetes server stopped working.

How we fixed it:

We redeployed new certificates which re-enabled the required communications.

What we are doing to prevent it from happening again:

We are undertaking a complete audit of all platforms to identify any other certificates that are due to expire. We are also implementing monitoring and alerting around certificates within all platforms to prevent this issue from recurring in the future.

Posted Feb 27, 2020 - 11:32 PST

Tests, tunnels and UI are running again. All services are now fully operational.
Posted Feb 22, 2020 - 13:58 PST
Our engineers have identified root cause and our services are recovering. We will continue to monitor closely.
Posted Feb 22, 2020 - 13:35 PST
Automated tests are also affected and can't be started. We continue to take remedial action.
Posted Feb 22, 2020 - 12:16 PST
In addition to our UI, REST API and Sauce Connect tunnels are not responding. We continue to investigate.
Posted Feb 22, 2020 - 11:47 PST
saucelabs.com and app.saucelabs.com are not available. We are investigating.
Posted Feb 22, 2020 - 11:41 PST
This incident affected: Sauce Connect (Sauce Connect VM), Web Interface (Sauce UI, Analytics, saucelabs.com), Automated VM Testing (Automated PC Testing, Automated Mac Testing, Automated iOS Simulator Testing, Automated Android Emulator Testing), and REST API (REST API VMs).