Penetration test, vulnerability scan, security audit or red team?

Four different services are sold under overlapping names. A scan gets called a penetration test, a configuration review gets called an audit, and 'red team' gets attached to anything with a phishing email in it. Buying the wrong one is rarely dangerous in itself; believing it answered a question it never asked is.

Four assessments, four different questions

Vulnerability scan: which known weaknesses are exposed?

An automated tool compares what it can see against a database of known vulnerabilities and misconfigurations. It is fast, repeatable and broad, and it should run continuously against anything you expose to the internet. It cannot reason about your business logic, and it cannot tell that user A should never see user B's invoice.

Security audit: do our controls meet a standard?

A reviewer examines configuration, code and evidence against a defined benchmark, such as OWASP ASVS (opens in a new tab) for applications (version 5.0 since May 2025) or the CIS Benchmarks for operating systems, databases and cloud services. An audit is not adversarial. It reads the settings an attacker would have to guess at, which is why it finds problems that testing from the outside never reaches. See our website security audit, database security audit and application security audit.

Penetration test: can someone get in, and how far?

NIST describes penetration testing as security testing in which evaluators mimic real-world attacks to find ways around the security features of an application, system or network, often by combining vulnerabilities to gain more access than any single one allows. In practice that means a time-boxed, mostly manual engagement against a defined scope, where the tester chains weaknesses together to prove impact rather than listing them. Our penetration testing page explains how an engagement runs.

Red team: would we notice, and how fast would we respond?

A red team emulates a realistic adversary against the whole organisation, staff included, and tests whether your detection and response notice. NIST's glossary frames the purpose as improving security by demonstrating the impact of successful attacks and showing defenders what works. Success is measured in objectives reached and time to detection, not in a count of findings. A purple team runs the same techniques collaboratively, with defenders tuning detections while they watch, usually mapped to MITRE ATT&CK (opens in a new tab) techniques.

What each one finds, and what it misses

A scanner touches everything shallowly; a red team goes as deep as it can along one path. The UK's National Cyber Security Centre says a penetration test should give you assurance that your vulnerability management works, and is not meant to be your primary way of finding vulnerabilities. Scanning and audits find the bulk; testing proves whether what is left can hurt you.

Nine ways the four assessments differ

Payment-card rules draw the same line. The PCI Security Standards Council's penetration testing guidance, written for PCI DSS 3.2, describes a scan as identifying, ranking and reporting vulnerabilities at least quarterly and after significant changes, and a penetration test as a mainly manual process that identifies ways to exploit them, at least annually and after significant changes. Check the current version of the standard for your exact obligations, and see Compliance for how other frameworks treat testing.

Who should buy which, and when

The right first purchase depends less on company size than on what you have to protect and who is asking for assurance.

• Pre-revenue, no customer data yet. A threat model of what you are building and a review of secure defaults will do more than any test, because a flaw designed out never needs a retest.
• First enterprise customers or a funding round. An authenticated web application test of the product those customers will use, with a retest and an attestation letter. Run a scanner on your external estate in parallel; it is the baseline, not the assurance.
• Regulated data or a certification on the horizon. An audit against the relevant standard, then a penetration test of what the audit cannot prove: that the controls hold under attack.
• A security team, a SIEM and on-call responders. Now a red team or purple team exercise tells you something new: whether the people and detections you invested in work under pressure.
• After a significant change. A new authentication system, a move to a new cloud account or a major API version deserves a targeted test of the change, not a wait for the annual cycle.

One rule holds at every stage: do not commission a red team before you have fixed what a penetration test would find. An adversary emulation against unpatched, internet-facing systems ends quickly and teaches little. Regulators increasingly name the assessment they expect, too. The EU's Digital Operational Resilience Act includes requirements for threat-led penetration testing in financial services, and the European Central Bank published an updated TIBER-EU, its framework for threat intelligence-based ethical red-teaming, in February 2025 to align it with DORA's technical standards for those tests.

How they combine over a year

For a growing SaaS company, a year usually runs in this order:

1. 1

   Always on: scanning

   Weekly or continuous scans of the external estate, plus dependency and secret scanning in CI. Triage the results into the backlog like any other defect.

2. 2

   Early in the year: threat model and audit

   Threat model the riskiest features on the roadmap, then audit cloud, database and application configuration against your chosen benchmark. Fix the broad, low-effort problems first.

3. 3

   Before the big release or the annual review: penetration test

   Test the product the way customers use it, after the audit has removed the noise, so tester time goes on what only a person can find.

4. 4

   Within weeks: retest

   Prove the fixes hold, and collect the attestation letter your customers will ask for.

5. 5

   After significant change: targeted tests

   Test the parts that changed rather than waiting for next year's cycle.

6. 6

   Once the basics hold: red or purple team

   Exercise detection and response against a realistic objective, then turn what you learned into next year's monitoring improvements.

Questions to ask any provider

• Can I see a redacted sample finding, with its reproduction steps?
• How much of the engagement is manual testing, and what exactly do the automated tools do?
• Which standard does the audit measure against, and which version of it?
• How is severity scored, and how is our business context applied to the score?
• Is a retest included, and do we receive an attestation letter afterwards?
• Who will do the work, and what are their qualifications?
• What happens if you find a critical issue, or signs of a real intruder, in the middle of the test?

If you are still unsure which assessment fits, a scoping call is the fastest way to find out. Sometimes the honest answer is that you need an audit or a threat model before any test, and we will say so.