AWS的验尸报告~

这里有个简版。

http://internet.solidot.org/internet/11/04/30/0915200.shtml

发现异常为啥不能及时回滚呢?其实我好奇这个

@ry_wang 他们错误的回滚策略造成了雪崩，引起更大范围的错误~

@xatest 哦?有详细点的说明么?官方文档看不懂呀

@ry_wang 这还不够详细？5700字了~

看着有种心惊肉跳的感觉

两点感受
1. 人为的操作容易出现错误。之前碰到过hosting的人员在配置Firewall的时候误操作，Firewall Hang住，导致网站无法访问。
The traffic shift was executed incorrectly and rather than routing the traffic to the other router on the primary network, the traffic was routed onto the lower capacity redundant EBS network.

2. 对回滚操作的后果估计不足，带来连锁雪崩效应。除非对系统各个模块和它们之间的交互有足够的理解，说不定就会踩中地雷。
Incorrect traffic shft was rolled back and network connectivity was restored. This quickly led to a “re-mirroring storm,” where a large number of volumes were effectively “stuck” while the nodes searched the cluster for the storage space it needed for its new replica.