# One-Off Penetration Test vs Continuous ECU Security Testing

> A one-off penetration test is a deep, independent look at an ECU at one point in time, delivered as a report. Continuous security testing is a repeatable suite that runs on every build or release and keeps a dated history: enumeration, SecurityAccess checks, fuzzing, DoIP and SOME/IP, with PDF and SARIF per run. The pentest finds what only a person finds; continuous testing makes sure it stays fixed. ISO/SAE 21434 treats vulnerability analysis as an ongoing activity, which a report from one date cannot satisfy alone.

A one-off pentest describes the ECU on one date; continuous testing watches every firmware release. What each delivers for ISO/SAE 21434 and UN R155, and how to combine.

Source: https://auto-st.com/compare/one-off-pentest-vs-continuous-testing · Updated: 2026-10-07

## What is the difference?

A one-off penetration test answers "how secure is this ECU today?" An engineer gets the sample, the documentation and a few weeks, finds what can be found and writes it down. The report is thorough and independent, and it describes exactly one firmware version on one date. Everything that happens afterwards, the fix that reopened a service in the default session, the new variant that shipped with a different seed/key configuration, the supplier update that re-enabled XCP, is outside its knowledge.

Continuous security testing answers "is this ECU still as secure as it was?" The same suite of checks runs on every build or release: sessions and services, DIDs, SecurityAccess counters and seed randomness, a seeded fuzzing run, DoIP routing, SOME/IP exposure, IVI hardening. Each run leaves a dated report and a SARIF file, risk acceptance carries over, and a regression shows up the day it is introduced. What it cannot do is think: it finds the known classes and the regressions, not the new attack path.

ISO/SAE 21434 Clause 8 describes vulnerability analysis and management as continual activities across the lifecycle, and UN R155 asks the manufacturer's CSMS to cover development, production and post-production. A report from one date is a snapshot inside that; the regulation describes a process.

## Side by side

| | One-off penetration test | Continuous security testing |
|---|---|---|
| What it finds | Logic flaws, seed/key algorithms recovered from firmware, bootloader and update weaknesses, chained attacks across interfaces, anything novel, on one firmware version | Regressions in sessions, services and DIDs, SecurityAccess counters and seed randomness, crashes and hangs under fuzzing, DoIP routing and SOME/IP exposure, IVI hardening, on every version |
| What it misses | Everything after the report date; regressions; variants not in scope | Logic, context, chains, firmware internals; anything without a known pattern |
| Who does it | External or internal security engineers with a scope and a deadline | Your test engineers, triggered by the pipeline or a schedule |
| Bench time | Two to six weeks per engagement | Minutes to hours per run, unattended, on every build |
| Cost | 20,000 to 45,000 euros per ECU, typical range for Germany and the EU, not an offer | Licence per tested component, price on request; the bench you already have |
| Fits a release cycle | No, it is a milestone: per platform generation, before start of production, after major changes | Yes, this is what it is for |
| Required by | ISO/SAE 21434 Clause 11 [RQ-11-01] names penetration testing for validation; OEM requirement sets | ISO/SAE 21434 Clause 8 continual activities and Clause 10.4.2 [RC-10-12]; UN R155 CSMS across the lifecycle; UN R156 for every software update |
| Output | A report with method, findings, reproduction, severity and fixes; a debrief and a retest | Dated scan history, PDF and SARIF per run, a Fix Plan with status carried across runs, JUnit XML in the build |

## Choose a one-off penetration test when

- The platform is new or has changed fundamentally: new microcontroller, new bootloader, first Ethernet interface, new security concept. Someone has to understand it before a suite can protect it.
- Validation is due. Before start of production and for the type-approval file, an independent test with a named tester is what Clause 11 and most OEM requirement sets mean.
- The risk is in logic and design: who may unlock which level, what the update path trusts, how the gateway routes. Automation does not know what should be allowed.
- You have a suspicion that needs a person: a strange routine, a diagnostic service that behaves differently per variant, a seed that looks random but may not be.

## Choose continuous testing when

- The ECU is released more than once. From the second firmware version on, a report from the first one is history, and someone has to check that the fixes held.
- Many variants share a platform. Running the same suite across all of them finds the one where a supplier left the programming session open.
- Evidence must be traceable. An assessor asking "what did you test on this version, and what happened to the findings" is answered by scan history, SARIF and a Fix Plan with status, not by a PDF with last year's date.
- You want a gate in the pipeline: fail the build when the risk score crosses a threshold, with JUnit XML where the build results already are.
- Nobody on the team is a security specialist. A suite that test engineers run from a dashboard is how the checks happen at all.

## Both together

The pattern that works is one deep test per platform generation and continuous testing in between. The pentest sets the baseline and finds what tools cannot; the continuous suite turns the pentest findings into tests that run on every release and keeps new regressions out. When the next pentest is due, the tester gets the history and spends the days where they matter. The report stops expiring because the process around it does not.

AutoST is the continuous half. It runs the repeatable checks on every build via API or Bamboo plugin, keeps the dated history and exports PDF and SARIF; Zyberum's own penetration testers are the other half, and we do not pretend the product can do their job. If an ECU is released once and never updated, a single pentest is enough and nobody needs continuous testing for it.

## FAQ

**Is a pentest report from last year still valid evidence?**

It is valid evidence about last year's firmware. If the ECU has had a release since, the report describes a product that no longer ships. Assessors know this and ask what has been done since; a dated scan history that shows the pentest findings closed and no regressions is the answer they expect.

**How often should the continuous suite run?**

Enumeration and SecurityAccess checks are fast enough for every build. A short seeded fuzzing run fits every build too; a long campaign belongs on each release candidate. The point is that the frequency is set by the release cycle, not by the budget for external testers.

**Does continuous testing make the pentest unnecessary?**

No. It makes the pentest better and less frequent. The tester gets the scan history, skips what automation already covers, and spends the days on firmware, bootloader and chained attacks. ISO/SAE 21434 Clause 11 still names penetration testing for validation.

## Sources

- [ISO/SAE 21434:2021 Road vehicles, Cybersecurity engineering, Clause 8 (continual cybersecurity activities: monitoring, vulnerability analysis and management) and Clause 11](https://www.iso.org/standard/70918.html)
- [UN Regulation No. 155, paragraph 7.2 (CSMS requirements over development, production and post-production)](https://unece.org/transport/documents/2021/03/standards/un-regulation-no-155-cyber-security-and-cyber-security)
- [UN Regulation No. 156, Software update and software update management system](https://unece.org/transport/documents/2021/03/standards/un-regulation-no-156-software-update-and-software-update)
- [OASIS SARIF 2.1.0 (Static Analysis Results Interchange Format)](https://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html)

## Related

- [Automated ECU Security Testing vs Manual Penetration Test](https://auto-st.com/compare/automated-ecu-testing-vs-manual-pentest)
- [SARIF (Static Analysis Results Interchange Format)](https://auto-st.com/glossary/sarif)
- [Traceability in an ISO 21434 audit: making fuzzing count](https://auto-st.com/insights/traceability-iso-21434-audit)
- [AutoST for CI/CD Engineers: ECU Security Scans as a Pipeline Gate](https://auto-st.com/for/ci-cd-engineers)
- [Security tests on every build, not once a year.](https://auto-st.com/ci-cd-integration)
- [Evidence for your 21434 work, on tap.](https://auto-st.com/iso-21434-testing)

---
AutoST by Zyberum. Canonical page: https://auto-st.com/compare/one-off-pentest-vs-continuous-testing