AutoST for ISO/SAE 21434 Compliance Managers: Test Evidence You Can Trace

The job

The ISO/SAE 21434 compliance manager, often called cybersecurity manager or CSMS owner, is the person who answers the auditor. The role owns the cybersecurity plan, makes sure the TARA, the cybersecurity concept and the verification reports exist for each item, keeps the interface agreements with suppliers in order and assembles the cybersecurity case. The engineers do the testing; the compliance manager has to prove that the testing happened, was sufficient and led somewhere.

This is where most programmes are weakest. When we supported a UN R155 and ISO/SAE 21434 audit at a Tier-1 supplier, the testing itself was not the problem. The problem was that test results lived in spreadsheets, e-mails and pentest PDFs with different structures, and nobody could say quickly which test covered which cybersecurity requirement, on which software version, and when it last ran. Building that table by hand took weeks before the audit.

What the role needs from a test run

The compliance manager needs evidence that answers four questions without a meeting: what was tested, when, against which requirement, and what happened to the result.

• A dated report per component. Each AutoST scan produces a PDF with an overview, the findings and a non-repudiation footer. The scan history per component shows that the test was repeated after each change, not done once.
• Traceability to requirements. Every Fix Plan task carries an ALM/PLM reference field that links the finding to the cybersecurity requirement or the ticket. Exports in CSV, JSON and SARIF 2.1.0 go into the tooling the organisation already uses.
• Documented risk decisions. Risk acceptance and comments are recorded per finding and carried across re-scans, so an accepted residual risk remains documented with its reason and a fixed issue shows as resolved.
• A consistent score. The weighted risk score from 0 to 10 is configured per tenant to the organisation’s own risk model, so it can be referenced in the TARA review rather than argued about.
• An honest scope statement. What AutoST tested (UDS services, SecurityAccess, fuzzing, DoIP, SOME/IP, Android IVI) and what it did not (FlexRay, LIN, hardware, secure boot), so the evidence is not mistaken for a penetration test.

Typical setup

Compliance managers rarely touch a bench. They get a dashboard role that shows them the components, scans, reports and the Fix Plan of the groups they are responsible for. The security or test team runs the scans with the Carbyne agent on the bench, over CAN, CAN FD, DoIP or SOME/IP, and owns the test profiles.

Most organisations that care about audit evidence run AutoST on their own servers, so reports and findings stay inside the document control they already have; the instance hosted by Zyberum in the EU works as well. AutoST has no connector to DOORS, Polarion or codebeamer: the link is the reference field and the export.

A sensible first project

Take one item that is due for an audit or a supplier review and build its evidence chain end to end.

1. Week 1: map requirements to tests. List the cybersecurity requirements of one ECU that are testable at the diagnostic and bus interface: locked services, SecurityAccess levels, robustness against malformed input, gateway routing. Agree with the test team which AutoST scan covers each one.
2. Week 2: baseline. The test team runs enumeration, SecurityAccess checks and the DID and routine scan. Enter the requirement IDs in the ALM/PLM reference field of each resulting Fix Plan task.
3. Weeks 3 and 4: fuzzing and decisions. Seeded fuzzing runs; every finding is fixed, assigned or accepted with a written reason.
4. Week 5: retest. The same profile runs again after fixes. The second report shows what was closed.
5. Week 6: mock audit. Walk an internal reviewer from a requirement to the test, the report, the finding and the decision, and time how long it takes.

How results feed ISO/SAE 21434 and UN R155

AutoST covers the testing clauses, not the whole standard. ISO/SAE 21434:2021 Clause 10.4.2 requires verification of the implementation against the cybersecurity specification ([RQ-10-09]) and recommends component testing with fuzz testing and vulnerability scanning ([RC-10-12]); the AutoST reports and scan history are the evidence for that work product. Clause 11 ([RQ-11-01]) names penetration testing for validation; AutoST is the baseline for it, not a substitute. Clause 8.5 makes vulnerability analysis a continuous activity, which repeated scans after every release support. Clause 7 governs distributed activities with suppliers; asking a supplier to deliver the result of an agreed AutoST profile is a practical way to fill the interface agreement.

UN R155 paragraph 7.2 requires the manufacturer’s CSMS to cover testing and supplier risks, paragraph 7.3 requires appropriate and sufficient testing for the vehicle type, and Annex 5 lists the threats and mitigations to address. For ECU-level threats such as unauthorised diagnostic access, AutoST provides repeatable evidence that the mitigation works. UN R156 adds that every software update must keep that security, which means the same profile runs again after each update. AutoST is not a certification and does not make anything compliant; it makes the testing part of the cybersecurity case traceable.