HackerOne has added two improvements that increase vulnerability report quality for public disclosure and bug bounty programs: Signal Requirements and an updated Rate Limiter. Signal Requirements allow a company to set a Signal threshold that hackers must reach in order to submit reports to them. We also updated our Rate Limiter to provide hackers the opportunity to still participate in a limited way, even if they are below the Signal requirement. If the hacker meets the Signal requirement, they may continue to participate.
One of the main challenges companies face in running a public program at scale is managing noise, or the proportion of low value reports they receive. HackerOne decided to focus on improving public program performance for two primary reasons. First, the more that HackerOne enables programs to continuously scale, the more they generate bounty opportunities for hackers. Second, programs that can scale beyond private programs into public ones improve their defense. This is a win-win.
HackerOne has worked continuously to reduce noise on the platform for our customers. We’ve shared some of these efforts in previous posts. Check out introducing Hacker Reputation, expanding Hacker Reputation and Signal improvements made over the first 10,000 bugs resolved on HackerOne. The total impact of our efforts has resulted in cutting down platform-wide noise by about half so far.
Although we’ve made improvements, there still remains a noise level of 28% on the platform. In other words, 28% of all reports received by customers are categorized as “not applicable” or “spam.” Studying the sources of these remaining invalid reports, we found that 50% of this noise is generated by just 10% of the hackers on the platform. Signal Requirements and our updated Rate Limiter serve to further reduce this remaining noise.
Let’s walk through how teams can set Signal Requirements.
The above screenshot shows how companies can set a required Signal that hackers must meet in order to participate in their programs. There are four settings that range from turning off any minimum required Signal threshold, all the way up to tightly limiting hacker participation based on high Signal (i.e., “Strict” setting). A strict setting makes sense for teams that prefer fewer, higher quality reports or that can only handle a smaller flow of reports. Turning Signal Requirements down or off is better for teams that value having the maximum number of hackers helping find issues as well as have the resources to manage the volume.
Our improved Rate Limiter works in combination with Signal Requirements. It provides all hackers the opportunity to participate in a program, even if their Signal doesn’t meet the program requirement. In this case, hackers will be given a number of trial reports which allows them to still report important issues if they find them. Previously, Rate Limiter either allowed hackers to submit many reports, or a capped them at just a few reports a day, depending on their Reputation. The Rate Limiter’s improved logic is now more nimble, raising and lowering the number of reports that may be submitted based on the hacker’s changing performance. This way, as hackers improve, the number of reports they can submit increases.
The new Signal Requirements and updated Rate Limiter improve the quality of reports programs can expect in public programs. At the same time, it allows all hackers to participate in the process of bringing important vulnerabilities to the attention of companies.
Programs that remained private solely to control noise now have the ability to transition to public while still maintaining high signal. More eyes on your attack surface help your team discover and address more critical issues. And now you can have the best of both worlds: more hackers helping you, with less noise.
If you have any feedback, thoughts or questions about Signal Requirements and the Rate Limiter, we’d love to hear from you. As always, please feel free to contact us at [email protected].
HackerOne is the #1 hacker-powered security platform, helping organizations find and fix critical vulnerabilities before they can be criminally exploited. As the contemporary alternative to traditional penetration testing, our bug bounty program solutions encompass vulnerability assessment, crowdsourced testing and responsible disclosure management. Discover more about our security testing solutions or Contact Us today.