Matt Atkinson |
21 January 2022 at 14:40 UTC
When we launched Burp Collaborator back in 2015, PortSwigger deployed a public Collaborator server that anyone could use. This meant that OAST testing with Burp Collaborator was able to work straight out of the box, with zero configuration. Automated OAST especially, was a game-changer – bringing many hard to find vulnerabilities to the masses for the very first time.
As you can see from the diagram below, all Burp Collaborator servers currently follow a monolith-type model – storing data in memory (RAM):
The problem
The problem with this is that it isn’t exactly scalable. Since 2015, Burp Suite Professional’s user base has increased by an order of magnitude (from around 8,000 in mid 2015 to over 60,000 in early 2022) – putting increased pressure on the public Burp Collaborator server.
Compounding this situation, in 2018 we introduced Burp Suite Enterprise Edition, which enables scalable, automated scanning of entire web portfolios. Given that Burp Suite Enterprise Edition uses the same Burp Scanner (and Collaborator) found in Burp Suite Professional – and at the time of writing is being used by 800 organizations – this adds considerably to the work being done by Burp Collaborator’s public server.
Then consider that these pressures only became more pronounced with the advent of the recent Log4Shell vulnerability found in the Log4j library. OAST is perfect for detecting this type of vulnerability – so naturally a lot of you are using tools like Log4Shell Scanner to check for this. Again, this looks to have had a marked effect on the amount of traffic going to the public Collaborator server.
Essentially, Burp Collaborator’s server had become a victim of its own success. The time was ripe for an update – and fortunately we’d been working on that for a little while.
The solution
We’ve now re-architected the public Burp Collaborator instance with a new design. This is cloud-native, elastic, and will greatly increase resilience. While users shouldn’t notice any functional difference, this will allow us to keep offering you a reliable public Collaborator server – at no extra cost on top of your Burp Suite subscription fee.
As you can see from the diagram below, we’ve introduced a number of changes in the new architecture. These changes allow for multiple instances of Collaborator’s services, running behind load balancers. This change necessitated the addition of a database – which we have implemented using Redis. The new architecture greatly increases horizontal scalability, as well as ensuring that we can upgrade the public Burp Collaborator server without worrying about downtime or loss of user data:
While re-architecting, we took the opportunity to design a system that provides Collaborator data with an enhanced level of protection while at rest. Consider the diagram below. Here you can see how any interaction data generated by Collaborator is encrypted, before being stored. Both a user secret and a master secret are then required in order to decrypt this data:
We’ve always taken the security of Collaborator data very seriously, and as such, Burp Collaborator has never stored any data that could be used to identify any individual Burp Suite user (such as an account name or license key). This is still very much the case.
In order to decrypt any interaction data, the Collaborator polling service requires both the user secret (given to it by a user’s instance of Burp Suite) and the master secret. Once your data has been returned, it is immediately deleted from the database – and should an instance of Burp Suite not request stored interaction data within 14 days of creation, then that data will be deleted from the database as stale.
Private Burp Collaborator servers
Of course, the option to deploy a private Burp Collaborator server still exists. This is particularly useful for those with large-scale or particularly sensitive use cases, where full control of the Collaborator server is required.
Currently, private Collaborator servers all use the original monolith-type model – but we’re also considering open sourcing the new elastic model for use on private Collaborator servers. If you have a relatively large use case for Burp Collaborator at the service provider level, and would like to help out with testing this functionality, then we’d like to hear from you. Drop us an email, and we’ll keep you updated.