Wait times are intermittently spiking to 100 or more seconds on all clouds. These spikes only last about three minutes and occur about once every two hours. We are investigating the cause.
2020-February-21 Service Incident
Incident Report for Sauce Labs US West Data Center
Postmortem

Dates:

Friday February 21st 12:36 am - 05:54 am PT

What happened:

Extended Team Management customers were unable to start new Sauce Connect tunnels originating in our San Jose, CA datacenter during the affected period.

Why it happened:

A configuration change broke communication between two services. The change affected all Extended Team Management users attempting to create new Sauce Connect tunnels. As a result of the configuration change, our internal tunnel monitoring failed to alert us to this condition.

How we fixed it:

We corrected this issue by reverting to the previous stable service version immediately after detecting a problem. This resulted in the rapid recovery of the system.

What we are doing to prevent it from happening again:

Additional monitoring is being put in place to consistently validate new tunnel connections for Extended Team Management users.

Posted Feb 26, 2020 - 16:10 PST

Resolved
We have taken remedial action. Sauce Connect tunnels are starting normally and all services are fully operational.
Posted Feb 21, 2020 - 05:58 PST
Identified
Sauce Connect tunnels are not starting in our US-West data center. We are taking remedial action.
Posted Feb 21, 2020 - 05:45 PST
This incident affected: Sauce Connect (Sauce Connect VM).