The Cisco Webex outage: how an ex-employee wiped 456 servers 5 months after quitting
Five months after he resigned from Cisco, a former engineer got back into the company's cloud and deleted 456 virtual machines, the servers that ran its Webex Teams collaboration app.[1][3] This case file covers what happened on that day in September 2018, what it cost Cisco and its customers, and the one control that would have made the whole thing impossible.
What happened
Sudhish Kasaba Ramesh worked at Cisco for nearly 2 years before resigning in April 2018.[1][4] Webex Teams, the company's chat and meeting app for businesses, ran on Cisco's cloud infrastructure hosted by Amazon Web Services.[1]
On September 24, 2018, Ramesh accessed that cloud environment without authorization. Working from a personal Google Cloud account, he deployed code that deleted 456 virtual machines supporting Webex Teams.[1][3]
The effect for customers was immediate. More than 16,000 Webex Teams accounts were shut down, some for as long as 2 weeks, while Cisco rebuilt what had been erased.[1][2] Prosecutors said no customer data was compromised; the harm was the outage and the cleanup.[1]
How he got in
He did not need to trick anyone. According to prosecutors, he simply reached into Cisco's cloud account from outside the company months after his last day, and his access still worked well enough to deploy code and delete servers.[1][3]
Public court records did not explain exactly which credentials or permissions let him back in, and press coverage noted it was unclear how a former employee could still reach the infrastructure.[2] What is clear is the shape of the gap: an account, key or permission that should have died with his employment was still alive in April and still alive in September.
Prosecutors also did not publicly describe a motive. In his plea, Ramesh admitted he acted recklessly in deploying the code and consciously ignored the obvious risk that it would harm Cisco.[1]
What it cost
Cisco spent about $1.4 million in employee time restoring the damage and refunded more than $1 million to affected customers, for a total of about $2.4 million.[1][4]
Ramesh was indicted in July 2020 and pleaded guilty on August 26, 2020, to one count of intentionally accessing a protected computer without authorization and recklessly causing damage.[1][5] On December 9, 2020, a federal judge in San Jose sentenced him to 24 months in prison, 1 year of supervised release and a $15,000 fine.[1] The charge carried a maximum of 5 years.[3] He was ordered to report to prison in February 2021.[1]
For a single afternoon's work by one person, that is a large bill: 2 weeks of customer outages, millions in costs and a federal prison sentence, none of it involving sophisticated hacking.
The missing control
The missing control: revoking every bit of cloud access on the day someone leaves. Offboarding usually means collecting a laptop and shutting off an email account. Cloud consoles, access keys, shared service accounts and project permissions often live somewhere else and get forgotten.
If every account, key and role tied to Ramesh had been cut in April 2018, there would have been nothing for him to use in September. And if Cisco had been reviewing who could still reach its production cloud, a working login belonging to someone who had left 5 months earlier would have stood out as an obvious problem long before it was used.
What to do in your business
- Write a one-page offboarding checklist. List every system a person might touch: email, cloud accounts, website host, domain registrar, accounting, payroll, social media, shared passwords. Work through it on their last day, not the week after.
- Change shared passwords when people leave. Any password or access key that a departing employee or contractor knew should be changed the same day.
- Review access every quarter. Pull the user list from each important system and compare it with your current staff. Anyone who is gone, or no longer needs it, comes off.
- Use one login to rule the rest. Where you can, have staff sign in to business apps with their company email account, so disabling that one account shuts most doors at once.
- Keep backups the cloud admin cannot delete. Store copies of critical systems and data somewhere separate, so a single person with the wrong access cannot wipe out both the live system and its recovery copy.
Check your business for this control
The free Heist Control Checklist walks through the controls behind every case on this site in about ten minutes. For ready-made policies, the Policy Pack has five editable templates, and the Insider Threat Kit covers risks from inside your own team.
- Guide: Employee offboarding checklist: how to cut off a former employee's access the same day
- Guide: How to verify callers before password resets, and protect your phone number from SIM swaps
- Guide: Which two-step login actually stops phishing, and how a small office rolls it out
- How the SEC's X account was hacked: a fake ID, a SIM swap and no MFA
- How the Colonial Pipeline hack happened: one unused VPN account and no MFA
- How the 2020 Twitter hack happened: a fake help desk call and an admin tool
- How the Mirai botnet knocked Twitter and Netflix offline with factory passwords
- How the Ronin bridge was hacked: a fake job offer and $600 million
- How Uber got hacked in 2022: a stolen password and one tired tap
- Every identity and access control
- All case files
Facts are drawn from court records, government reports, company statements and reputable reporting, listed below. People are named only where they were convicted, pleaded guilty or spoke publicly in an official role.
- U.S. Attorney's Office, Northern District of California: San Jose man sentenced to two years imprisonment for damaging Cisco's network
- Threatpost: Ex-Cisco employee convicted for deleting 16K Webex accounts
- BleepingComputer: Ex-Cisco engineer who nuked 16K WebEx accounts goes to prison
- Infosecurity Magazine: Former Cisco engineer gets two years for $2.4M WebEx attack
- SecurityWeek: Former Cisco employee sentenced to prison for Webex hack