Q&A: From Modular Software to End-to-End AI
ADAS historically has used collections of specialized software components to perceive the environment, track objects, plan maneuvers and control vehicle motion. End-to-end AI takes a different approach: Rather than treating driving as a sequence of discrete functions, it trains a single model to learn driving behavior directly from data. Because it is so different, the approach raises important questions about development, validation, scalability and deployment. In this Q&A, Guillermo Pita, Aptiv vice president of engineering for autonomy, AI and ML, explains how end-to-end AI systems work and what they could mean for the future of assisted and automated driving.
How does an end-to-end AI architecture for ADAS differ from a traditional, modular system?
Traditional ADAS breaks the driving task into separate functions. One component identifies vehicles, pedestrians, lane markings and other objects. Another tracks them. Others perform sensor fusion, planning and vehicle control.
With an end-to-end AI system, we teach the model to drive based on examples, rather than explicitly programming each step. Instead of building a chain of individual modules and handoffs to decide “Is it safe to turn left?” the model learns the relationship between what it sees and the action it should take.
The approach parallels human driving. No one consciously calculates trajectories and distances every time they make a turn. They assess the overall situation and make a decision based on experience and context, evaluating the driving situation holistically rather than following a series of explicit rules.
Hasn’t ADAS used AI and machine learning for years?
Yes, but typically within individual functions rather than across the full driving task. For example, Aptiv has been applying machine learning to cameras, radar perception and path planning for many years. The difference with end-to-end AI is the holistic approach.
What are the advantages and disadvantages of end-to-end AI?
The biggest advantage is more human-like driving. Instead of optimizing perception, planning and control individually, end-to-end AI optimizes the entire driving task. That often produces behavior that feels more natural, because the system learns directly from real driving examples.
The challenge is explainability and debugging. In traditional architecture, it’s usually possible to determine exactly where a problem originated in perception, tracking or planning, and therefore fix it directly. In an end-to-end system, it can be more difficult to understand exactly why a model behaved a certain way. Improvements often come from better training data and model refinement rather than fixing a specific software module.
What are the biggest challenges in training driving models?
While everyone talks about the amount of data — and that’s important — data diversity is really the key challenge.
A system needs exposure to different road types, weather conditions, traffic patterns, infrastructure and driving behaviors. A model trained primarily in one part of the world may perform well there yet struggle elsewhere because the driving environment is different.
The goal isn’t simply collecting more miles. It’s collecting the right miles and exposing the model to as many meaningful situations as possible. For example, consider an ambulance approaching an intersection. While the correct response may seem obvious, these situations vary significantly depending on traffic, road layout, weather and local driving behavior. They also occur infrequently. Training a model to reliably handle these relatively rare cases requires diverse and carefully selected data.
How can companies generate that much training data?
Real-world driving data remains essential, but developers also use simulation and synthetic scenarios to supplement rare events. These tools make it possible to expose models to situations that occur infrequently on public roads and evaluate system behavior across a broader range of conditions
Edge cases are where the real challenge begins.
Normal driving is relatively straightforward. What’s difficult are the unusual situations that happen infrequently: a roadway configuration the model rarely encounters, unexpected behavior from another road user, or transportation systems that are unique to a particular region.
You can’t anticipate every possible scenario, but you can improve the system’s ability to generalize by exposing it to a broad range of environments during training. The more diverse the training data, the better the model becomes at responding appropriately when it encounters something new.
How do end-to-end AI systems approach sensor fusion?
End-to-end AI systems still use multiple sensors, including cameras and radar, because each has different strengths.
For example, radar is more useful than cameras in poor lighting and visibility conditions, such as nighttime or fog. Those sensing technologies are complementary rather than competing. Using both allows us to build systems that perform more consistently across a wide range of environmental conditions where relying on a single sensing technology may introduce limitations.
The difference is in how the information is combined. In a traditional architecture, engineers explicitly define many of the sensor-fusion processes. In an end-to-end AI system, the model learns the relationships among different sensor inputs and how those inputs contribute to driving decisions under different conditions. Through training, it learns patterns that allow it to make use of information from multiple sensing modalities.
How do centralized vehicle computing architectures support end-to-end AI?
End-to-end AI fits naturally with the industry’s move toward centralized computing. Instead of distributing processing across many separate controllers, sensor data can be brought together in a central computing platform. That gives the model access to all the information it needs while reducing the number of processing steps between perception and action.
Centralized architectures also provide the computational resources required to run complex AI models.
That said, every architectural decision involves trade-offs. Centralized architectures can improve efficiency and reduce latency. However, they also concentrate more functionality on a single platform. That makes reliability, fault management and system design even more important.
What will make an end-to-end AI system successful?
End-to-end AI represents a significant change in how driver assistance systems are developed, trained and deployed. While the approach introduces new challenges around data collection, scaling and model development, it also offers a path toward more adaptable and natural driving behavior.
For the automotive industry, success depends both on AI technology advances and on an organization’s ability to translate millions of miles of driving experience into systems that perform consistently and safely across the wide variety of roads, environments and conditions drivers encounter every day.
End-to-end AI is not just a model development challenge. It’s a data, validation, deployment and continuous-learning challenge. Organizations that close that loop effectively are best positioned to succeed.