How to read a penetration test report (and argue about severity)

A penetration test report is written for two audiences at once: the people who decide what to fund, and the people who write the fix. Most readers skim the severity chart, panic or relax, and never check the vector behind a score or the root cause shared by three similar findings. To see the format first, there is a real finding, anonymised on our work page.

What a complete report contains

Section names vary between providers, but a complete report answers the same questions. The PCI Security Standards Council publishes a report outline and an evaluation checklist in its penetration testing guidance, and both make a useful yardstick for any report, payment-related or not.

Two quality signals are easy to spot. First, findings should be demonstrated rather than inferred: the PCI guidance itself says that merely reporting lists of vulnerabilities does not meet the intent of a penetration test. Second, the narrative should say which manual and automated testing was done and what held up, because a report that lists only failures gives you no idea how much ground was covered.

Severity: what CVSS 4.0 measures, and what it leaves out

Most reports score findings with the Common Vulnerability Scoring System. Version 4.0 was published by FIRST on 1 November 2023 and organises its metrics into four groups.

Scores fall into five bands: None (0.0), Low (0.1 to 3.9), Medium (4.0 to 6.9), High (7.0 to 8.9) and Critical (9.0 to 10.0). A score should always arrive with its vector string, which begins CVSS:4.0/ and lists every base metric. Without the vector you cannot check the reasoning, only the conclusion.

Two defaults matter more than any other detail. In CVSS 4.0 an unset exploit maturity is scored as if the vulnerability is already being attacked, and unset security requirements are scored as High. A Base-only score, written CVSS-B, is therefore a reasonable worst case, and FIRST's own user guide says Base scores measure severity and should not be used alone to assess risk. Threat and Environmental metrics, written CVSS-BT, CVSS-BE or CVSS-BTE, are where real-world context belongs.

That is why our reports pair a CVSS 4.0 score with an adjustment for your business context, and why you should expect any provider to show the reasoning behind an adjusted score. For findings that are known CVEs in third-party components, two public sources add evidence about exploitation: CISA's Known Exploited Vulnerabilities catalogue (opens in a new tab), which records vulnerabilities exploited in the wild, and FIRST's EPSS (opens in a new tab), which estimates the probability that a published CVE will be exploited in the next 30 days.

How to argue about severity

The tester knows how the attack works; you know what the system is for, so expect to dispute some ratings. The UK's National Cyber Security Centre says vulnerability risk assessment is a business process that should not be wholly outsourced to the test team. The goal of the conversation is the right priority, not a smaller number.

Arguments that should move a score

• A compensating control that exists and can be shown to work, such as network policy that genuinely blocks the attack path.
• A precondition that cannot occur in your deployment, for example a feature flag that is off for every customer.
• Data that is less sensitive than the tester assumed, backed by your data classification.
• Evidence that the affected component is not reachable in production at all.

Arguments that should not

• It is hard to find, or nobody would think to try it.
• Our scanner did not flag it.
• We plan to rewrite that service next quarter.
• The attacker would need an account, when anyone can sign up for one.
• A higher rating would delay a release or a customer deal.

How to dispute a rating

1. 1

   Restate the finding

   Describe the attack in your own words and confirm with the tester that you have understood it.

2. 2

   Name the metric you dispute

   Point at the specific part of the vector, such as attack requirements or privileges required, rather than at the overall number.

3. 3

   Bring evidence

   Configuration, logs or architecture that supports a different value. Assertions are not evidence.

4. 4

   Re-score in the open

   Agree the new vector and record the rationale in the report. The PCI guidance asks for a traceable set of reasoning whenever a standard score is modified, and an auditor or customer will want to follow it later.

5. 5

   Fix the priority, not the label

   If you accept a lower severity, still decide when it will be fixed. Accepting a risk should be a named person's decision, with a date to review it.

Reproduction steps and evidence

A finding you cannot reproduce is a finding you cannot fix or verify. Good reproduction steps read like a recipe, with:

• The account and role used, and any set-up the attack depends on.
• The exact requests or actions, in order, with the parameters that were changed.
• What should have happened, and what happened instead.
• Evidence: request and response captures, screenshots or output, with sensitive data redacted.
• Timestamps and source addresses, so you can find the attack in your own logs.

That last item is a free detection exercise. Search your logs and alerts for each finding's timestamps: if a critical authorisation bypass left no trace your team would have seen, you have learned something about monitoring as well as about code, and your incident response readiness. If your engineers cannot reproduce a finding within an hour or so, raise it in the walkthrough rather than closing it as unconfirmed. The difference is usually an account state or a header the steps took for granted.

What good remediation guidance looks like

Generic advice such as 'validate all input', or a bare link to an OWASP page, tells an engineer which category of thing went wrong, not what to change. Good guidance is specific to your stack and fixes the class of bug, not only the instance.

Look for three things in each remediation section: the root cause, the specific change, and a way to verify it. When findings share a root cause, a good report says so. Three insecure direct object references are usually one missing authorisation pattern, and fixing the pattern is worth more than patching three endpoints. If the same class keeps returning, the fix is partly in how the team builds, which is what secure development training addresses.

Retests and attestation letters

A retest confirms that fixes work and did not introduce anything new. It should cover every finding you report as fixed and produce an updated report rather than an email. The PCI guidance suggests a retest report carry the dates of the original test and the retest, the original findings and the result for each, which is a sensible minimum from any provider. Expect every finding to end in one of a few states:

• Fixed: the tester can no longer reproduce it.
• Partially fixed: the original instance is closed, but a variant still works.
• Not fixed: the reproduction steps still succeed.
• Risk accepted: you decided not to fix it, with a named owner and a reason.

An attestation letter is a short document for third parties: who tested what, when, using which methodology, and the status of the findings after retest. It is what you send a customer's security team instead of the full report, which is a map of how to attack you. It is not a certification or a guarantee, and a careful reader will check its dates and scope against the product they are buying. Our engagements include a free retest of every fixed finding within 30 days of report delivery and a reissued report with an attestation letter, and anything we have not verified as fixed stays open in the letter.

Briefing the board and briefing engineers

The same report needs two translations, and neither is a copy of the executive summary.

For directors who want a structured way into the subject, the NCSC publishes a Cyber Security Toolkit for Boards (opens in a new tab). For engineers, hold the walkthrough within days of the report arriving, while the tester's memory and your context are both fresh. Most severity disputes are settled in that room, in front of the evidence, and most remediation questions are quicker to answer there than in a ticket thread.

If you are about to commission a test and want a report that reads like this, start with a scoping call, or read how our penetration testing engagements run from scope to retest.