In this Help Net Security interview, Scott Sutherland, VP of Research at NetSPI, delves into the intricacies of their Breach and Attack Simulation (BAS) platform and discusses how it offers unique features – from customizable procedures to advanced plays – that help organizations maximize their ROI.
This interview also explores the crucial role of Key Performance Indicators (KPIs) in tracking the efficacy of security measures. It gives us a sneak peek into NetSPI’s real-time dashboards that are set to change how security professionals interact with data.
Can you provide a high-level overview of NetSPI’s Breach and Attack Simulation platform and what makes it unique?
We deliver a centralized detective control platform that allows organizations to create and execute customized procedures utilizing purpose-built technology and professional human pen-testers. Simulate real-world attack behaviors, not just IOCs, and put your detective controls to the test in a way no other organization can.
Can you speak to how organizations can visualize ROI through the NetSPI platform?
Breach and Attack Simulation solutions should help provide ROI in a variety of ways:
- BAS solutions should provide data insights into where your detective and preventative control gaps are so you can make intelligent choices about where to invest your security dollars. This should include point-in-time and overtime reporting to justify or validate investments meaningfully. For example, this should include visualizations showing how investments in new data sources can increase alert coverage for common attack behaviors. Another typical example would be visualizing the increase in detection rule coverage results from adding another detection engineer.
- Recruiting, training, and educating pentest and SOC teams can take time and money. Most BAS tools should include educational material that your teams can use to understand how to execute and detect common attack behaviors within the application. This can save both time and money in the long run.
- There are hundreds, if not thousands, of hacker tools. Researching, installing, and running them to simulate the newest malicious behavior can be time-consuming and risky if the mechanisms are better understood. BAS solutions can take that off your team’s plate so they can focus on doing the job of simulation, detection engineering, and control validation/tuning.
- Finally, tracking the average ransomware trends can help people estimate the potential cost of the ransomware incidents that BAS solutions are designed to help prevent and detect.
How does the platform allow organizations to build, configure, and run customizable procedures? Could you walk us through a typical use-case scenario?
The NetSPI BAS platform provides many customization features to meet everyday use cases.
Plays can be used in the platform to automate the execution of expected behaviors and procedures real-world attackers use. All of which have customization options.
Advanced Plays provide users with additional flexibility, so if they can dream it, they can do it. The advanced play packs offer users the means to do many things, including, but not limited to, dropping payloads, executing arbitrary commands, executing arbitrary code, and even replaying packet captures through deployed agents.
Playbooks allow users to execute plays in any sequence they would like. This will enable them to simulate TTPs for a specific threat actor or perform behaviors required for their purple or red team use cases.
What real-world attack behaviors can be simulated using the NetSPI platform?
We’ve spent much time researching threat actors’ attack procedures at each phase of the MITRE ATT&CK Kill chain and other frameworks. Then, we’ve taken those lessons learned and leveraged them to prioritize the development of the procedures and plays we put into the platform so they are more meaningful to our clients. As a result, our clients can simulate a myriad of real-world threat actor behaviors and help them prepare for the next wave of hacker groups that will likely be using some of the same “hacker trickers” as their predecessors.
Could you discuss the role of KPIs in tracking and trending the effectiveness of security controls?
Many SOC teams lean heavily on their average response time as one of their primary KPIs. However, they don’t measure if their technology stack provides the monitoring/alerting coverage needed to detect standard malicious behavior in their environment. In many cases, the response team may only be as good as the alerts in their dashboard. So, one of their KPIs should be tied to the question, “Are the right alerts showing up?”. In the absence of knowing what that benchmark should be, many companies end up unquestioningly trusting their security vendors without validation. BAS solutions should be able to provide meaningful metrics for tracking if common malicious behaviors are making it to your IR dashboards.
Whether you’re tracking global coverage or the level of coverage your vendors provide, we recommend paying attention to the following:
1. Number of typical TTPs coverage by MITRE ATT&CK phase over time, not just at the technique level, but at the procedure level. Also, track each procedure’s logging, alerting, prevention, and response ticket levels.
2. Number of common TTPs commonly associated with ransomware over time, not just at the technique level, but at the procedure level. Also, track each procedure’s logging, alerting, prevention, and response ticket levels.
3. The number of new procedures introduced over time based on threat intelligence.
Can you delve into the real-time dashboards? What kind of information do they provide, and how actionable is the data for security professionals?
The Platform provides you with the ability to create operations that can be organized by system, region, business unit, and more. Reporting can be scope to those operations to provide a point-in-time view of where your detection coverage is today and how it has improved over time. We are also adding vendor report cards that will be coming later in the year so you can dig into the performance of your security vendor investments.
What advice do you have for organizations considering implementing breach and attack simulation into their security posture?
Ensure you understand what you are trying to get out of our Breach and Attack Simulation solutions and come to your vendor discussion with a solid list of requirements you can work through. This will help ensure alignment and get what you need from the service or product.
Below are some of the common goals we hear from clients:
- We want to understand where our detection gaps are so we can make better choices when investing in staffing and security vendor dollars.
- We want to understand how much detection/prevention coverage we have for our standard asset classes, regions, and business units.
- We want to help educate and enable our internal red, purple, and detection engineering team through in-application TTP.
- We want better, more meaningful KPIs to enable and empower our SOC.
- We want a better way to evaluate our security vendors.
- We want to perform ongoing control validation for those detection/preventative controls we already have.