Google periodically devotes a week to Disaster Recovery Testing (DiRT).

Netflix automates resilience testing per-job using Chaos Monkey, and per-cluster using Simian Army.