How a North Carolina System Survived a Ransomware Attack

In March 2020 the city and county of Durham, North Carolina were hit by a ransomware attack that crippled all their IT services and facilities, including their computer-aided dispatch system, 9-1-1, and internal email. It was a startling situation that forced first responders to improvise communications solutions on the fly.

The history and fallout of that ransomware attack were covered in the FirstWatch webinar “Conversations That Matter—Durham Ransomware Attack 2020”. It is available to watch here. (According to www.firstwatch.net, “FirstWatch turns raw data into meaningful information, helping agencies improve situational awareness, operational performance, and clinical patient outcomes. Our system does this by securely capturing, translating, and transmitting information about their 9-1-1 callers, patients, and systems via FirstWatch triggers, all in real time.”)

Leslie O’Connor was chief emergency manager for the Durham County Office of Emergency Services when the ransomware attack took place (she is now enterprise crisis manager for Labcorp, a Fortune 500 health science company). During the webinar O’Connor shared the strategies she and her colleagues used to help first responders stay in service despite the loss of their IT-backed communications system. Here are some coping strategies they used to make this happen.

Work the problem—As soon as the city and county became aware their IT systems had been crippled by ransomware, the key people involved with IT and emergency services got together to “work the problem.” Right off the bat they focused on questions such as, “What resources do we have available?” recalled O’Connor. “What contracts had they already paid for that could help with information security and being able to figure out what exactly happened? Who could help us figure out who was patient zero and how far this had actually spread?”

Get alternative comms in place fast—Aware that some form of first responder communications needed to be in place while dispatch and everything else was down, the city and county of Durham “set up a department operation center at their headquarters,” O’Connor said. “All night they listened to the radio and were monitoring who was on call, what their status was, and what their current location was.”

Armed with this data, the staff kept a running tally of where their public safety vehicles were and what they were doing, writing the details on a dry erase board.

Adapt where you can—The ransomware attack was confined to Durham’s IT systems; the general internet was left unscathed. So were popular applications such as WhatsApp and Google Maps. All three were combined to provide an ad hoc dispatch system for the affected area that could be accessed by first responders’ unaffected smartphones.

“One of the lieutenants created a WhatsApp group for the different shifts,” explained O’Connor. “They would get the address from the call taker, and they would send the link to Google Maps via WhatsApp. They gave them [the responders] the call type and [other] basic information over WhatsApp so they were able to respond to that call…as seamlessly as they could.”

Coming up with this sort of workaround was a big help for first responders, especially as their “CAD was down for probably a month,” she added. “A lot of that had to do with trying to put security mechanisms in place that may not have been present to start with, so we wouldn’t be impacted again as hard as we were.”

The Cloud Made a Difference

As bad as it was, the ransomware attack could have been worse.

“Fortunately, we [had] put up the good fight in advance [of the attack] with technology governance boards and requested that our emergency operation center software that we use for incident management was stored in the cloud offsite and did not need to be authenticated through Durham County,” said O’Connor. The county’s Everbridge mass notification system and BOLDPlanning continuity of operations plans were also stored in the cloud, shielding all of them from the attack. This meant all of them remained available to use.

“We did utilize the Everbridge emergency notification system to contact all our employees in both the city and county to let them know that…when they got to work on Monday morning, they were not to try to use their devices at all,” she said.

Any Laptop in a Storm  

The ransomware attack knocked out any city and county computers connected to the network via ethernet cables, including laptops. In response to this loss, employees besieged their IT people for replacements that worked.

Of course, there was no warehouse full of backup laptops in place to fulfill this need. Fortunately, a former city IT employee had since moved over to Dell, so Durham leaders were able to call them for help. But Dell wasn’t the only company contacted. “We called in every resource we possibly knew that could help us with non-networked, straight-out-of-the-box functional laptops,” said O’Connor. “That included our neighbors in Person County. Their board of elections…was able to give us about 35 laptops as loaners.

“And so we started distributing laptops, and all was well and good until we overloaded the Verizon network with our [mobile wireless hotspots],” she continued. “Very quickly we made a phone call to FirstNet through AT&T and said, ‘We’re in trouble and could really use your help.’ So they sent us about 45 Netgear Nighthawks (mobile hotspots) with the AT&T FirstNet SIM card. We were able to get those distributed to the people who needed them most.”

These are just some of the ransomware survival tactics O’Connor shared, all of which proved a key point: When ransomware shuts down a first responder IT system, creative thinking, along with using whatever technology still works, can allow police, fire, and EMS to keep serving their communities.

James Careless is a freelance writer and frequent contributor to EMS World.