Estimated financial losses due to the recent massive IT outage triggered by the faulty CrowdStrike update are counted in billions, but the unfortunate incident is having several positive effects, as well.
Some silver linings
As CrowdStrike was forced to explain, in great detail, how they roll out updates for its Falcon Sensors, what testing they perform beforehand, and how they plan to improve the whole process to prevent similar accidents from happening in the future, other cybersecurity vendors – such as Fortinet, Secureworks, and Bitdefender – have spelled out their own software and content update release processes.
Hopefully, they are also taking this opportunity to re-evaluate whether additional improvements are warranted.
David Weston, VP of Enterprise and OS Security at Microsoft, has penned a post explaining why security vendors leverage a kernel driver architecture: for system-wide visibility, to detect bootkits and rootkits, for faster data collection and analysis, and for tamper resistance.
“Kernel drivers provide [those] properties at the cost of resilience,” he explained. “All code operating at kernel level requires extensive validation because it cannot fail and restart like a normal user application.”
But, he noted, security tools can minimize kernel usage while still maintaining a robust security posture and strong visibility.
“For example, security vendors can use minimal sensors that run in kernel mode for data collection and enforcement limiting exposure to availability issues. The remainder of the key product functionality includes managing updates, parsing content, and other operations can occur isolated within user mode where recoverability is possible,” he said, and outlined Windows’ user mode protections vendors can use to protect their key security processes and maintain event visibility.
Finally, he also explained how Microsoft tests and signs drivers, the alternative ways for third-party vendors to distribute them to users (e.g., via Windows Update), and spelled out the company’s intent to:
- Help third-party vendors safely update their products
- Work with them to reduce the need for kernel drivers to access important security data, and
- Provide enhanced isolation and anti-tampering capabilities in the Windows OS.
The scope of the outage
Based on publicly available data, UpGuard has compiled a list of companies that have been affected by the CrowdStrike Falcon incident – and it’s long.
Microsoft’s recent estimate of the number of systems (8.5 million) thrown in a Blue-Screen-of-Death loop by the defective update is based on the number of crash reports/dumps the company received from customers who choose to share than information, Weston said.
On Thursday, CrowdStrike said that over 97% of its Windows Falcon Sensors were back online. The rest may be on Windows systems that have yet to be restored, and some of the sensors may have already been removed in a knee jerk reaction to the incident.
In the meantime, threat actors have been exploiting the chaos and jumping on every pretext – including the botched CrowdStrike apology gift cards – to defraud, infect, and misinform the public.