One-Off Penetration Test vs Continuous ECU Security Testing

What is the difference?

A one-off penetration test answers “how secure is this ECU today?” An engineer gets the sample, the documentation and a few weeks, finds what can be found and writes it down. The report is thorough and independent, and it describes exactly one firmware version on one date. Everything that happens afterwards, the fix that reopened a service in the default session, the new variant that shipped with a different seed/key configuration, the supplier update that re-enabled XCP, is outside its knowledge.

Continuous security testing answers “is this ECU still as secure as it was?” The same suite of checks runs on every build or release: sessions and services, DIDs, SecurityAccess counters and seed randomness, a seeded fuzzing run, DoIP routing, SOME/IP exposure, IVI hardening. Each run leaves a dated report and a SARIF file, risk acceptance carries over, and a regression shows up the day it is introduced. What it cannot do is think: it finds the known classes and the regressions, not the new attack path.

ISO/SAE 21434 Clause 8 describes vulnerability analysis and management as continual activities across the lifecycle, and UN R155 asks the manufacturer’s CSMS to cover development, production and post-production. A report from one date is a snapshot inside that; the regulation describes a process.

Side by side

	One-off penetration test	Continuous security testing
What it finds	Logic flaws, seed/key algorithms recovered from firmware, bootloader and update weaknesses, chained attacks across interfaces, anything novel, on one firmware version	Regressions in sessions, services and DIDs, SecurityAccess counters and seed randomness, crashes and hangs under fuzzing, DoIP routing and SOME/IP exposure, IVI hardening, on every version
What it misses	Everything after the report date; regressions; variants not in scope	Logic, context, chains, firmware internals; anything without a known pattern
Who does it	External or internal security engineers with a scope and a deadline	Your test engineers, triggered by the pipeline or a schedule
Bench time	Two to six weeks per engagement	Minutes to hours per run, unattended, on every build
Cost	20,000 to 45,000 euros per ECU, typical range for Germany and the EU, not an offer	Licence per tested component, price on request; the bench you already have
Fits a release cycle	No, it is a milestone: per platform generation, before start of production, after major changes	Yes, this is what it is for
Required by	ISO/SAE 21434 Clause 11 [RQ-11-01] names penetration testing for validation; OEM requirement sets	ISO/SAE 21434 Clause 8 continual activities and Clause 10.4.2 [RC-10-12]; UN R155 CSMS across the lifecycle; UN R156 for every software update
Output	A report with method, findings, reproduction, severity and fixes; a debrief and a retest	Dated scan history, PDF and SARIF per run, a Fix Plan with status carried across runs, JUnit XML in the build

Choose a one-off penetration test when

• The platform is new or has changed fundamentally: new microcontroller, new bootloader, first Ethernet interface, new security concept. Someone has to understand it before a suite can protect it.
• Validation is due. Before start of production and for the type-approval file, an independent test with a named tester is what Clause 11 and most OEM requirement sets mean.
• The risk is in logic and design: who may unlock which level, what the update path trusts, how the gateway routes. Automation does not know what should be allowed.
• You have a suspicion that needs a person: a strange routine, a diagnostic service that behaves differently per variant, a seed that looks random but may not be.

Choose continuous testing when

• The ECU is released more than once. From the second firmware version on, a report from the first one is history, and someone has to check that the fixes held.
• Many variants share a platform. Running the same suite across all of them finds the one where a supplier left the programming session open.
• Evidence must be traceable. An assessor asking “what did you test on this version, and what happened to the findings” is answered by scan history, SARIF and a Fix Plan with status, not by a PDF with last year’s date.
• You want a gate in the pipeline: fail the build when the risk score crosses a threshold, with JUnit XML where the build results already are.
• Nobody on the team is a security specialist. A suite that test engineers run from a dashboard is how the checks happen at all.

Both together

The pattern that works is one deep test per platform generation and continuous testing in between. The pentest sets the baseline and finds what tools cannot; the continuous suite turns the pentest findings into tests that run on every release and keeps new regressions out. When the next pentest is due, the tester gets the history and spends the days where they matter. The report stops expiring because the process around it does not.

AutoST is the continuous half. It runs the repeatable checks on every build via API or Bamboo plugin, keeps the dated history and exports PDF and SARIF; Zyberum’s own penetration testers are the other half, and we do not pretend the product can do their job. If an ECU is released once and never updated, a single pentest is enough and nobody needs continuous testing for it.