Have you ever wondered how we manage to test your website for hundreds of vulnerabilities without making you wait too long? We have built a powerful fingerprinting algorithm to spend just the right amount of time to find what we look for, and this is how we do it.
More and more vulnerabilities come to the surface every day, so it would take an increasingly long time to check for all of them against every single website. Many of these vulnerabilities are also very specific and can only affect certain web servers or Content Management Systems (CMSs). It would therefore be very time consuming and ineffective to test blindly for all of them.
We do the very best in order to keep our scanner up-to-date with the latest security threats, from the most generic to the platform-specific ones. When we test a website we do not just scroll a long list of possible vulnerabilities trying to find all those who affects it, but we rather tailor our tests to the technology stack that we find.
Having a clear picture of what web servers, CMSs and libraries types and versions run on a website is not at all a trivial task. It involves a cautious evaluation of the website content and of the messages exchanged between clients and server. Our so-called fingerprinting algorithm (see detectify.com/technology) at the core of the Detectify scanner, is where all this magic happens.
All the information collected by our crawler is fed to a classification algorithm that is able to decide within a bunch of millisecond what web servers, CMSs and libraries we are dealing with. On the basis of that information, we are then able to start looking for all pertinent vulnerabilities, excluding those that we know for sure are not there.
//Andrea Palaia
About Andrea:
Andrea is a data scientist at Detectify. He moved to Sweden from Italy in 2009 for a Ph.D. in accelerator physics, and for several years he has been jumping back and forth between CERN, Uppsala and Berlin. After his Ph.D. he started to pry into the startup world with Patamu.com and about 8 months ago he landed at Detectify where he makes numbers speak.