Communications of the ACM
Ban 'Naked' Braces!
Credit: iStockPhoto.com
One fine business afternoon early in 1990, when we still used wires and microwave towers to make phone calls, and almost all long-distance calls went through big AT&T switches, one of the 100 or so 4ESS switches that handled U.S. long-distance traffic at the time hit a glitch and executed some untested recovery code. The switch went down briefly. No biggie, since traffic automatically took other routes, but in the process the initial switch that hit the glitch dragged its neighboring switches down, and the process cascaded across the country, as all the switches that handled long-distance traffic began to repeatedly crash and auto-recover. The result was that hardly any public telephone customer in the U.S. could make a long-distance phone call that afternoon, along with millions of dollars of time-sensitive business lost.
AT&T tried to contain the damage by rebooting the misbehaving switches, but as soon as a switch was brought back up, a neighboring switch would tell it to go down. The engineers at AT&T's R&D arm, Bell Labs, who wrote the switch programs, were called in, and, by the end of the day, network normality was restored by reducing the network message load.
No entries found