Google Cloud’s so-called uninterruptible power supplies caused a six-hour interruption

Google has revealed that a recent six-hour outage at one of its cloudy regions was caused by uninterruptible power supplies not doing their job.

The outage commenced on March 29th and caused “degraded service or unavailability” for over 20 Google Cloud services in the us-east5-c zone. Google’s US east zone is centered on Columbus, Ohio.

Google’s incident report states that the outage started with “loss of utility power in the affected zone.”

Hyperscalers build to survive that sort of thing with uninterruptible power supplies (UPSes) that are supposed to immediately provide power if the grid goes dead, and keep doing so for a few hours before diesel-powered generators kick in.

Google’s UPSes, however, suffered a “critical battery failure” and didn’t provide any juice. They also appear to have prevented power from generators reaching Google’s racks, because the incident report states the advertising giant’s engineers ...