In-House ECU Security Testing vs an External Test Lab

What is the difference?

In-house testing means the ECU, the bench and the engineer are yours. You decide when a test runs, you see the result the same afternoon, and nothing about the firmware or the diagnostic description leaves the building. The limits are expertise and independence: your team knows your ECU but may not know what attackers do to other people’s ECUs, and a supplier assessing its own product is not what an OEM means by validation.

An external test lab is independent by construction. It brings people who test ECUs from many suppliers all year, hardware you do not own (climate chambers, bus analysers for every transport, side-channel and fault-injection rigs where needed) and a report that an OEM or assessor accepts as third-party evidence. The limits are time and cost: a lab engagement is a project with a scope, a start date and a deliverable, and the ECU you send is a snapshot. Six weeks later there is a new firmware and the report describes the old one.

Side by side

	In-house testing	External test lab
What it finds	Everything your tools and people cover, on every build: diagnostic surface, SecurityAccess weaknesses, robustness under fuzzing, Ethernet exposure, IVI hardening, regressions	Everything an experienced outsider finds on a snapshot: the above plus firmware-level weaknesses, chained attacks, hardware attacks, comparison with what other suppliers do
What it misses	Blind spots of a team that knows the product too well; the independence an assessor wants for validation	Regressions after the report; your internal context; anything out of scope or outside the booked weeks
Who does it	Your test and QA engineers, ideally with one security-minded owner	The lab’s security engineers, under an agreement with scope and rules of engagement
Bench time	Minutes to hours per run, whenever a build is ready	Two to six weeks per engagement including the report, plus shipping the samples
Cost	Bench hardware you mostly have, engineer time, a tool licence per component	Project price per engagement; ECU penetration tests typically 20,000 to 45,000 euros in Germany and the EU, not an offer
Fits a release cycle	Yes, this is the point of it	No; once per platform generation, before start of production and after major changes
Required by	ISO/SAE 21434 Clause 10.4.2 verification is the supplier’s own job; UN R155 CSMS asks the manufacturer to manage testing along the supply chain	Independence for validation (Clause 11) is asked for by many OEM requirement sets and by type-approval assessors in practice
Output	Scan history, PDF and SARIF per run, a Fix Plan your engineers work through	A signed report with method, findings, reproduction and recommendations; a presentation and a retest

Choose in-house testing when

• You release firmware more than once a year or ship an ECU in several variants. Each release needs the same checks, and a lab cannot be booked every sprint.
• Confidentiality is strict. ODX and CDD files, pre-series firmware and seed/key material stay on your own servers; nothing has to be shipped or uploaded.
• You want findings fixed while the developer still remembers the change. A result the same day is worth more than a better result in six weeks.
• Your team should learn. Engineers who run enumeration and read negative response codes every week become the people who design the next ECU better.
• The checks are the repeatable kind: sessions, services, DIDs, SecurityAccess counters, fuzzing robustness, DoIP routing, SOME/IP exposure. Tools do these well and a lab would use tools too.

Choose an external test lab when

• Independence is the requirement. A supplier’s own report rarely satisfies an OEM’s validation clause, and an assessor for UN R155 type approval will ask who tested.
• The platform is new and nobody in-house has attacked one like it. A first external pentest sets the baseline the in-house suite later protects.
• You need hardware or skills you will not build up for one project: firmware extraction from a locked microcontroller, fault injection, side channels, environmental testing.
• You have no bench yet. A lab engagement is also a way to learn what a bench for this ECU should look like.

Both together

The combination that works: an external lab tests each new platform once, deeply and independently, before start of production. The in-house bench runs the repeatable suite on every build from then on and keeps the lab’s findings from coming back. When the next lab engagement is due, the tester gets the in-house scan history and spends the days on what automation cannot see.

AutoST serves both sides of this. Suppliers and OEMs run it on their own benches; test labs run it on their customers’ ECUs to make the repeatable part consistent across engagements and hand over PDF and SARIF with the report. Zyberum’s own penetration testers are a lab in this sense, and we are clear that AutoST is the tool they use for the first day, not a replacement for the following weeks.