When somebody says "AI in automotive", they could mean one of at least three things that have nothing in common except the acronym. The perception stack that keeps a car in its lane. The assistant that answers a question about the climate control. The model that predicts which component is about to generate a warranty claim.
Different data, different failure consequences, different economics, different teams. I want to take them separately, because the most expensive mistakes I have seen came from applying the instincts of one to another.
1. The driving stack
The architectural argument here has genuinely moved. The classical approach was modular: perception, then prediction, then planning, then control, each a separable component with an interface you could test.
The shift is towards end-to-end learned systems, where a single network consumes sensor input and emits trajectories, trained on very large quantities of driving data. The motivation is real — hand-authored interfaces between modules discard information, and the planning stack accumulates thousands of heuristics that nobody can reason about collectively.
The safety-engineering cost is also real, and it is the part that gets waved away in demos. A modular stack lets you argue about the correctness of each piece. An end-to-end network gives you a system whose behaviour you can measure statistically but not decompose. That is not a reason to reject it. It is a reason the industry has been building out a different kind of assurance argument: enormous scenario libraries, simulation, and a shift from "we verified the component" to "we characterised the behaviour across a distribution".
The intermediate position most production programmes actually occupy: learned perception with a much richer intermediate representation — occupancy and semantics in a bird's-eye-view space rather than a list of bounding boxes — feeding a planner that is still partly explicit and therefore still constrainable. You get the information-preserving benefits without giving up the ability to impose a hard rule about, say, never crossing a solid line.
Three practical things matter more than the architecture choice:
- The data engine. Whoever can mine their fleet for the specific rare scenario, auto-label it, retrain and validate fastest, wins. This is a data infrastructure competition dressed as a modelling competition.
- Simulation fidelity. Neural reconstruction of real drives — rebuilding a scene from logged sensor data so you can perturb it and re-simulate — is what makes rare-event testing tractable.
- Safety of the intended function. The hazard is often not a malfunction. The system works exactly as designed and the design was insufficient for the situation. That is a different analysis from functional safety, and it needs its own budget.
2. The cabin
This is where large language models arrived in cars, and where the gap between demo and product is widest.
A voice assistant that can actually converse is a genuine improvement over the command-grammar systems that preceded them, which everybody hated and few used. But putting a model in a cabin surfaces constraints that cloud products never face:
| Constraint | Why it bites |
|---|---|
| Connectivity | The car goes through tunnels. The feature cannot simply stop working. |
| Latency | Conversational turn-taking has a budget of a few hundred milliseconds before it feels broken. |
| Compute | Whatever silicon shipped with the vehicle is what you have for a decade. |
| Liability | The assistant must not confidently instruct a driver to do something unsafe. |
| Privacy | In-cabin audio is among the most sensitive data a vehicle collects. |
The architecture that survives contact with these is hybrid and explicit about it: a small on-device model handling wake, intent routing and the vehicle-control cases that must work offline, with escalation to a larger cloud model for open-ended queries — and a UX that degrades honestly rather than pretending.
The vehicle-function integration is the part worth designing carefully. An assistant that can adjust the cabin temperature is calling into the vehicle's service layer, which means the same interface governance, authorisation and rate-limiting questions as any other API — except a mistake here moves a physical machine. This is where I would want a hard, non-learned allowlist between the model and anything actuating.
3. The parts nobody demos
The AI that pays for itself soonest in automotive is deeply unglamorous, and it lives in the tail of the business rather than the vehicle.
Warranty and quality analytics. Every OEM sits on decades of claims data, dealer notes and field telemetry. Finding the correlation between a build-date range, a supplier lot and a failure that will surface in eighteen months is worth an enormous amount, and it is a tractable problem with ordinary tooling.
Predictive maintenance on real telemetry. Not the brochure version. The version where you accept that your labels are weak, your fleet is heterogeneous, and a false positive costs a customer an unnecessary service visit — so precision matters more than recall, which inverts the usual instinct.
Manufacturing inspection. Vision on the line, catching defects earlier than a human station, in a setting where the environment is controlled and the ROI is directly measurable.
Engineering productivity. Which is the one I have spent the most time on lately: agents that take on parts of the delivery lifecycle — analysis, migration, test generation — inside a pipeline that still has to be auditable. In a regulated context the constraint is not model capability, it is provenance. Every artefact needs to be traceable to a decision a human approved.
The regulatory layer arriving now
Two threads converge and both change how you build.
Safety standards for AI in road vehicles are formalising what "sufficient" means for a learned component — moving the conversation from "our benchmark number is good" to documented evidence about data coverage, distributional assumptions and known limitations.
Broad AI regulation adds obligations around transparency and risk classification that land on in-cabin systems handling personal data and on anything making decisions about people.
The engineering consequence is the same in both cases, and it is worth internalising early: your training data provenance is a compliance artefact. Where each sample came from, what consent covered it, which model versions it trained, and what the resulting model was measured against. Teams that treated the data pipeline as an internal implementation detail are now retrofitting lineage into it, which is far more expensive than building it in.
What I would tell someone allocating budget
Be specific about which of the three you mean, because they do not transfer:
- The driving stack is a data-infrastructure and assurance investment with a long horizon and a very high floor. Do not enter it casually.
- The cabin is a product and UX investment where the differentiator is graceful degradation, not model size.
- The tail — warranty, quality, manufacturing, engineering — is where a mid-sized team gets measurable return inside a year.
Most organisations I have watched over-invest in the first, under-invest in the third, and judge the second on a demo given with perfect connectivity in a stationary vehicle.