A Commerce Department agency spent nearly $3 million and more than a year fighting off a non-existent malware infection, going so far as to trash $170,000 worth of computers and other equipment in what an auditor cast as a wild overreaction to a misunderstood threat.
An audit report describing the saga reads like a comedy of errors. Thanks to a series of miscommunications and what the report diplomatically describes as a technology staff that "lacked appropriate IT security skills," the U.S. Economic Development Administration went nuclear on a minor problem, eventually spending half its IT budget for last year attacking the phantom infection, according to a report released last month by the Commerce Department's inspector general.
The EDA is a 170-person agency that focuses on job growth and regional economic development across the United States. Its technology meltdown began in December 2011, when the Commerce Department's emergency IT team sent a warning to two of its agencies about an infection it detected within their building-wide network.
A follow-up note clarified that the infection affected just two computers. EDA "misunderstood" that message, according to the audit report. Believing it faced a widespread attack, it launched an all-hands-on-deck response that eventually involved four additional government teams, an outside cybersecurity contractor, and the complete shutdown of the EDA's email network.
The report's most jarring revelation is that the EDA brushed off its contractor's conclusion that the agency faced no significant threat and could solve its problem with some simple repairs. The EDA's chief information officer decided instead that the only way to be 100% safe was to physically destroy all of the agency's technology gear, including TVs, cameras, computer mice and keyboards.
The EDA set out to trash $3 million worth of equipment, stopping short of its goal only because it ran out of money. Meanwhile, it relied on the U.S. Census Bureau for loaned equipment and BlackBerry service.
The agency's eradication and clean-up efforts lasted 15 months and cost more than $2.7 million. In contrast, the other affected agency, the National Oceanic and Atmospheric Administration, eliminated the routine malware within weeks.
The audit report is fairly scathing, laying out a trail of puzzling decisions and pricey missteps. The government paid $4,300 to destroy equipment and spent $823,000 on its contractor's investigation of the non-existent infection.
"EDA's persistent mistaken beliefs resulted in an excessive response and ultimately unnecessary expenditure of valuable resources," the Office of the Inspector General wrote in its report.
The EDA says it has learned from its mistakes.
"We have already begun implementing many of the recommendations in the OIG report," an agency spokesman said. "We take the privacy and IT security of all our employees, grantees and other partners seriously, which is why the agency acted out of an abundance of caution."
Did anyone get fired for the debacle? The EDA's spokesman declined to comment on personnel matters, but the audit report refers to the IT official who oversaw the malware response as the agency's "current" CIO.
The report has a silver lining: Government watchdogs eventually caught on to the EDA's unnecessary panic and turned down a request for $26 million to fund further "recovery efforts." The agency's remaining laptops and computer mice are safe from the incinerator.
