AWS Wide Spread Outage 2021-12-07 COPY

This five hour outage made headings in the popular media:

https://www.ft.com/content/6eff4cae-5df7-4746-a068-d4f6a18ba285

"From angry Adele fans to broken robot vacuums: AWS outage ripples through US

Millions hit by suspension of services that use cloud computing, including Amazon deliveries Please use the sharing tools found via the share button at the top or side of articles. Copying articles to share with others is a breach of FT.com T&Cs and Copyright Policy. Email licensing@ft.com to buy additional rights. Subscribers may share up to 10 or 20 articles per month using the gift article service. More information can be found at https://www.ft.com/tour. https://www.ft.com/content/6eff4cae-5df7-4746-a068-d4f6a18ba285

It started early on Tuesday morning. Robot vacuums ceased sucking, WiFi cameras stopped watching and eager Tinder daters were left unable to “swipe right” on their smartphone apps.

An outage at Amazon Web Services, the cloud arm of Amazon, had rippled through the online economy, crippling services used by millions of people.

Among the most distraught were fans of the British singer Adele who had been hoping to snap up the first tickets to her upcoming Las Vegas residency.

“Due to an Amazon Web Services (AWS) outage impacting companies globally,” the ticket seller Ticketmaster explained, “all Adele Verified Fan Presales originally scheduled for today have been moved to tomorrow.”

The disruption highlighted the degree to which many of the internet’s most popular services rely on cloud computing infrastructure handled by a very small number of large companies.

According to Gartner, 80 per cent of the cloud market is handled by just five companies. Amazon, with a 41 per cent share of the cloud computing market, is the very biggest.

“They’ve had some very large outages,” said Servaas Verbiest from Sungard Availability Services, a company that provides “disaster recovery” for multiple cloud platforms. “What makes AWS more exposed is the sheer volume of business they have.”

Within Amazon itself on Tuesday, the unthinkable occurred: grounded delivery drivers were unable to load packages and deliver to customers’ doorsteps, just as the peak Christmas season begins to step up.

Drivers at multiple facilities across the country were sent home with pay. With little to do, many of them logged on to social media to enjoy the moment while it lasted — some dreading whatever workload may await once systems were back up and running.

An “impairment of several network devices” in one of its server regions — US-EAST-1 — was the “root cause” of the disruption, Amazon said in a message posted to the AWS status page, which monitors the operational health of its global network of interconnected computers.

Amazon did not comment on the disruption to its deliveries.

Business Insider quoted an internal memo detailing a flood of traffic from an “as yet unknown source”.

Publicly, the company logged the first issues at 9.37am US Pacific time on Tuesday morning, though users of affected services had complained of problems before then. By 3pm, AWS said it had been able to mostly restore service.

Several of the sites first affected appeared to have been able to reroute traffic to alternative servers. Whether or not outages created longer-lasting problems for companies depended on the degree to which executives prioritised diversifying their cloud computing providers, added Verbiest.

“If you’ve embraced the ecosystem, and you’ve got everything in AWS, you’re in a sit-and-wait scenario,” he said.

While high-profile outages can be a boon for competitors such as Google and Microsoft, Verbiest stressed the bar to switching service providers was high.

“It’s difficult to say that one outage is going to sway people to one cloud platform or another, because every cloud provider has outages. It’s just about how long are they and how do they resolve them when they happen?”

Recommended Inside BusinessRichard Waters Every company may soon be a cloud company

In November 2020, the US-EAST-1 region was also at the heart of an AWS outage affecting many of the same websites. In that case, a fault with an Amazon system ...