Penetration testing is a good baseline for evaluating system vulnerabilities and an industry best practice that supports routine security hygiene. Many companies also use traditional pentests to pass vendor assessments and meet compliance standards like HITRUST, SOC 2 and ISO 27001. But by themselves, pentests aren’t enough to demonstrate security effectiveness.
As companies stay competitive with more continuous software development life cycles, pentests are being reframed to support a holistic security strategy. The goal is to use pentesting to discover where weaknesses hide, where engineering approaches can be improved, and where risks to your most valuable assets can be reduced.
We recently hosted a webinar, Incorporating Pentesting in Your Overall Security Strategy, with heads of security from both Sumo Logic and Dropbox. They discussed how their teams are rethinking pentests to both improve pentesting impact and incorporate those findings into a holistic security strategy. What they’re striving for is more impact for the time, budget, and effort they’re putting in.
“We were running penetration testing as a compliance checkbox, but we weren’t finding significant things that would help our process, and that’s what we wanted,” said George Gerchow, Chief Security Officer at Sumo Logic. “I want a pentest that’s constantly running. That’s one of the areas of value we saw that was broken with traditional pentesting. So we got away from traditional pentesting and started running bug bounties.”
That shift in pentest thinking resulted in both companies moving pentesting from a periodic checkbox to a strategic component of their security efforts.
“Given what I’m getting out of pentesting, I’m getting something different out of bug bounties, which is that constant stream of discovery,” said Justin Berman, Head of Security at Dropbox.
As Dropbox and Sumo Logic moved from traditional pentesting to always-on, bounty-based pentests, it complemented their continuous bug bounty programs and automated efforts. But each plays a part in their overall security strategy.
“I want to have broader coverage and find things faster,” added Berman. “That starts with a shift towards automation, then a blended approach using pentesting, bug bounties, and automation. (Security) is a marathon, it’s not a sprint. So it’s really critical to build a culture of continuous improvement. The data you get out of pentesting is a huge part of that.”
A recent HackerOne poll of security leaders also reflects this trend of moving pentest expectations beyond simply a compliance checkbox and towards a focus on new surface layers of attack. To that end, half of the respondents indicated that the main reason their organization is conducting pentesting is to discover vulnerabilities.
While many businesses rely on pentests to meet compliance standards, an increasing number of security and engineering leads are looking to scale the effort beyond compliance to improve their software development lifecycle. Unfortunately, traditional pentests fail to meet the growing needs of evolving businesses. The cost of additional security vendors can add strain to engineering teams and can hamper speed to market, most notably for businesses with small teams who wear many hats. With over half of business leaders relying on pentests to find vulnerabilities across their assets, the traditional model falls short of improving security postures.
Where Traditional Pentests Fall Short
Pentests should be comprehensive in their scope, dig into the nooks and crannies of code, seek out obscure vulnerabilities, and bring a fresh perspective to your security efforts. That is why traditional pentests are broken: they offer static results with limited transparency. But vulnerability discovery should happen as close to real-time as possible. In addition, most pentesters are highly talented but usually have biases, and these can strongly inform the results, especially if a small pentesting team is used repeatedly.
A traditional one-size-fits-all pentest usually comes with a limited set of testers and approaches, which results in delayed results. Agile development means applications are being updated monthly, weekly, or even daily, so an annual or quarterly pentest isn’t going to offer realistic or reliable insights. In addition, the lack of a modern SaaS platform for real-time communication with researchers and instant visibility into discovered vulnerabilities makes it difficult to scale pentesting efforts.
Managing Cybersecurity with Human Intelligence
Hacker-powered pentesting helps scale your testing efforts because it has much faster turnarounds, brings in many researchers offering creative techniques, and adds flexibility in timing, pricing, and approaches. A recent Forrester Total Economic Impact Report found that hacker-powered pentests are over $40,000 less expensive than a typical pentest. In addition, they can easily integrate with your internal processes and provide real-time visibility into potential security gaps as each potential vulnerability is discovered.
“When you hire a traditional pentester, you’re going to have one or two people, that are gonna poke at you for weeks, maybe a month,” said Justin Berman, Head of Security at Dropbox. “It’s slower and less frequent, but it’s also that you’re just not gonna have enough different views on what bad could happen.”
Using a larger pool of hackers adds diversity of approach, specialized skills, and wider experience — all resulting in faster discovery of security gaps unique to your business or technologies. At HackerOne, we look at this with a three-pronged proactive approach: Response, Bounty, and Pentest.
The first step is to make it easy to report vulnerabilities in a coordinated fashion through a Vulnerability Disclosure Policy (VDP). HackerOne Response has revolutionized VDPs to make it easy to work directly with trusted hackers to find and resolve critical security vulnerabilities.
Adding bounty programs and pentests then helps to further secure your applications with continuous testing. The diversity of bounty program options (private, public, time-bound, and virtual or live events) makes it easy to start with a manageable program, and then ramp up gradually or create a pinpoint focus on specific assets or utilizing hackers with specific skills.
Sumo Logic turned to HackerOne after their pentesting reports kept coming back clean. They knew it couldn’t be because they’re perfect — it was because their pentesters kept looking in the same places. Sumo Logic initiated its first private bounty challenge in late 2017. In just 15 days, 5 hackers found 12 vulnerabilities that had been missed by earlier pen tests.
Sumo Logic then optimized its pentests and bug bounty programs by including the auditors during vulnerability review and remediation, which helped ensure compliance throughout the process. Their team also leveraged HackerOne managed services to triage reports as they arrived, effectively becoming an extension of Sumo Logic’s security team and decreasing response times to hackers.
The Trend Towards Vendor Consolidation
Prior to working with HackerOne, Sumo Logic had dozens of security and testing tools in use but they lacked the resources and infrastructure to test at scale. The complexity of multiple SLAs and processes then put the different business units in a stalemate. Consolidating security vendors provides a platform approach rather than a collection of disconnected point solutions.
“We’re putting less investment in point solutions and doubling down on our critical partners,” said George Gerchow, Chief Security Officer at Sumo Logic.
Our recent survey showed that while there is still a lot of uncertainty around business continuity, the COVID-19 pandemic has accelerated the trend of automating and reducing further spend on tooling and point solutions. A recent poll showed 22% of security leaders surveyed indicated they’re planning to consolidate their security vendors in the next 12-16 months.
As your organization shifts toward more automation and scale, continuous validation into the effectiveness of your security controls is paramount. Hacker-powered pentests help you reach security at scale with continuous testing provided by thousands of researchers with varying skills, experiences, and approaches.
Consolidating security efforts with fewer vendors also brings focus to those efforts. By eliminating the need to evaluate, deploy, and manage an array of disparate scanning and testing tools, your security team can refocus their time and skills on more business-critical priorities, while rapidly scaling the security of new applications and infrastructure.
Want to learn more? Check out our webinar Incorporating Pentesting into Your Overall Security Strategy, featuring Sumo Logic’s Chief Security Officer and Dropbox’s Head of Security.