The Difference Between a Vulnerability Assessment and a Penetration Test

• Language Matters
  >

• Clarified Definitions
  >

• A Physical Analog
  >

• The Exploitation Angle
  >

• Pentests Don’t Include VA
  >

• Summary
  >

There are many views on what constitutes a Vulnerability Assessment versus a
Penetration Test. The main distinction, however, seems to be that some
believe a thorough Penetration Test involves identifying
as many vulnerabilities as possible, while others feel that
Penetration Tests are goal-oriented and are mostly unconcerned with
what other vulnerabilities may exist.

I am in the latter group, and what follows is my argument for why you should
be too.

Language Matters
>

Language is important, and we have two terms for a reason. We already have
an (aptly named I might add) security test for compiling a complete list of
vulnerabilities, i.e. a Vulnerability Assessment. If there isn’t a
clear, communicable distinction between this test type and a penetration
test then we shouldn’t be using separate terms. Such a distinction does
exist, however, and it’s a crucial one.

Clarified Definitions
>

Vulnerability Assessments are designed to yield a prioritized list of
vulnerabilities and are generally for clients who already understand they
are not where they want to be in terms of security. The customer already
knows they have issues and simply need help identifying and prioritizing
them.

The more issues identified the better, so naturally a white box approach
should be embraced when possible. The deliverable for the assessment is,
most importantly, a prioritized list of discovered vulnerabilities (and
often how to remediate).

Penetration Tests are designed to achieve a specific,
attacker-simulated goal and should be requested by customers who are already
at their desired security posture. A typical goal could be to access the
contents of the prized customer database on the internal network, or to
modify a record in an HR system.

The deliverable for a penetration test is a report of how security was
breached in order to reach the agreed-upon goal (and often how to
remediate).

A Physical Analog
>

A good analog for this is a Tiger Team working for the government, like
Richard Marcinko
>
used to run with Red Cell. Think about what his missions were: things like
gain control of a nuclear submarine and bring it out into the bay.

So imagine that he’s getting debriefed after a successful mission where he
broke in through the east fence, and someone were to ask him about the
security of the western side of the building. The answer would be simple:

If the person doing the debrief were to respond with, “You didn’t check the
other fences? What kind of security test is it where you didn’t even check
all the fences?”, the answer would be equally direct:

The Question of Exploitation
>

Another mistake people make when discussing vulnerability assessments vs.
penetration tests is to pivot immediately to exploitation. The basic
narrative is:

This is incorrect.

Exploitation can be imagined as a sliding bar between none and full, which
can be leveraged in both vulnerability assessments and penetration tests.
Although most serious penetration tests lean heavily towards showing rather
than telling (i.e. heavy on the exploitation side), it’s also the case that
you can often show that a vulnerability is real without full exploitation.

A penetration testing team may be able to simply take pictures standing next
to the open safe, or to show they have full access to a database, etc.,
without actually taking the complete set of actions that a criminal could.
And vulnerability assessments can slide along this scale as well for any
subset of the list of issues discovered.

This could be time consuming, but exploitation doesn’t, by definition, move
you out of the realm of vulnerability assessment. The only key attributes of
a VA vs. PT are list-orientation vs. goal-orientation, and the question of
exploitation is simply not part of that calculation.

The Notion that Penetration Tests Include Vulnerability
Assessments
>

It’s also inaccurate to say that penetration tests always include a
vulnerability assessment. Recall that penetration tests are goal-based,
meaning that if you achieve your goal then you are successful. So, you
likely perform something like a vulnerability assessment to find a
good vuln to attack during a pentest, but you could just as easily find a
vuln within 20 minutes that gets you to your goal.

It is accurate to say, in other words, that penetration tests rely on
finding a one or more vulnerabilities to take advantage of, and that people
often use some sort of process to systematically discover vulns for that
purpose, but because they stop when they have what they need, and don’t give
the customer a complete and prioritized list of vulnerabilities, they
didn’t actually do a vulnerability assessment.

Summary
>

Vulnerability Assessment

• Customer Maturity Level: Low to Medium. Usually requested by
  customers who already know they have issues, and need help getting
  started.

• Goal: Attain a prioritized list of vulnerabilities in the
  environment so that remediation can occur.

• Focus: Breadth over depth.

Penetration Test

• Customer Maturity Level: High. The client believes their defenses
  to be strong, and wants to test that assertion.

• Goal: Determine whether a mature security posture can withstand
  an intrusion attempt from an advanced attacker with a specific goal.

• Focus: Depth over breadth.