Wait times are intermittently spiking to 100 or more seconds on all clouds. These spikes only last about three minutes and occur about once every two hours. We are investigating the cause.
2019-November-19 Service Incident
Incident Report for Sauce Labs US West Data Center


November 19th, 2019 18:47 -19:30 PDT

What happened:

Some customers experienced high wait times running tests in our PC, Mac, & Android clouds in our US-West datacenter.

Why it happened:

A hypervisor rebooted which caused a load balancer running on it to lose some configuration changes and become overloaded

How we fixed it:

We restored the configuration values to the proper setting and it recovered.

What we are doing to prevent it from happening again:

We have reviewed all systems with high connections loads and verified that their configurations are correct. We have also verified that these changes are in the proper deployment configurations and that they are persistent in all required systems.

Posted Dec 18, 2019 - 13:10 PST

Wait times are back to normal. All services are now fully operational.
Posted Nov 19, 2019 - 15:19 PST
We have taken remedial action and tests should be starting. We are monitoring closely.
Posted Nov 19, 2019 - 14:56 PST
Wait times on our PC Cloud are high. There may be errors while starting PC jobs. We are investigating.
Posted Nov 19, 2019 - 14:30 PST
This incident affected: Automated VM Testing (Automated PC Testing).