Amazon says automation bug caused massive AWS outage

Amazon has published a lengthy report about the outage that knocked numerous websites, services, apps and games offline on October 20. It all started with a bug in its automation software DynamoDB, where its AWS customers store their data, which then triggered more issues in its other systems that relied on the software.

As Amazon explains, DynamoDB maintains hundreds of thousands of DNS records and is supposed to be able to fix any issue automatically. But on October 20, the DynamoDB DNS management system suffered from a bug that resulted in an empty DNS record for Amazon's data centers in North Virginia. DynamoDB was supposed to repair the issue on its own, but it had failed to do so, prompting Amazon to fix the problem manually. While the issue was happening, all systems that needed to connect to DynamoDB couldn't and experienced DNS ...