Recently, I spent the afternoon with seasoned automotive and technology reporters talking about autonomous driving. Specifically how Artificial Intelligence (AI) plays a part in the development of autonomous vehicle technology (AVT).
At the end of the day, several of the reporters thanked me for helping them understand these complicated topics. One even said, “I’ve been writing about AI for ten years and have been thinking about it all wrong.”
For engineers, technology is at the heart of everything we do and even how we think. We break down complex technical issues into the simplest possible elements and then put them back together again. Sometimes just in our own minds – but it helps us understand how things work. Once we understand that, we can come up with ways to make things work better.
And AI is tool we can use to solve problems that can’t be solved using classical techniques. AI has been around for many years; but we now have cost-effective processors allowing us to solve highly complex problems that were virtually impossible to solve before.
The key for AI is training – and training means collecting massive amounts of data that can be used to help train the AI algorithms (neural nets) to perform their task. AI isn’t the right tool for every job, but AI can be used to make our systems much more capable and robust.
And that’s what Delphi is trying to do AI and our AVT. So how do we use AI for our automated vehicles? The answer is, very carefully. We use AI algorithms within what we call a deterministic framework: a set of over-arching policies and rules that govern how the vehicle will operate (for instance, you can set a rule that you will never drive faster than the speed at which the vehicle is able to stop within the free space around the car). Additionally, you don’t need AI to solve for vehicle dynamic behavior or to know what the speed limit is; these are things that can be explicitly defined or modeled. We use AI for functions such as object classification, behavior modification, trajectory optimization, and vehicle to environment socialization. This is where AI can really add value – solving problems where you simply can’t write enough rules to cover all of the situations or corner cases.
So where does all of the data required for training come from? The primary means is through the car itself; its eyes and ears, or its sensors. We call that ‘on-board.’ There are three main types of sensors – vision (cameras), radar and LiDAR. While some companies are using only one or two types of sensors, Delphi believes you need all three types of sensors because none of them are 100 percent perfect in all operating conditions. Together, however, they create redundancies which makes for a much more robust perception system. We believe this strategy is critical to safety, as well as to ensure the car can, and will, operate in any condition. These sensors take in more than 40 Terabytes of data an hour and AI plays a critical role in processing this data - identifying, classifying and predicting what is happening around the vehicle.
And so while we drive… and drive… and drive some more, we continually ask the question, ‘How do we refine the system? How do we make it more adaptable, flexible and more comprehensive?’
The answer lies in part, to the use of AI. But, before I go too deep, too fast, let’s start with the basics. AI is defined as ‘any device that perceives its environment and takes actions that maximize its chance of success at some goal.’ For autonomous vehicles, that means getting safely to a predetermined destination.
What does that all really mean in practical terms – and for our current development state, how do we manage all that data?
The cameras or vision systems were the first to use AI and are currently the heaviest users. Why? A camera alone doesn’t do much. It captures images, but it doesn’t know what those images are or how to react to them. That’s where AI and machine learning come into play.
You have to train the system - and this requires technicians to annotate the images our sensors collect so that they can be used to teach the system - ‘that’s a dog, a cat, a tree, a child, a street sign.’ The more images can be used to train the system, the better the system will perform. This is especially important for object classification where we are attempting to determine what exactly the camera systems are seeing.
As I said earlier, we don’t stop there because we believe you need to use all three sensors, which also include radar and LiDAR. A human can’t easily look at a radar or LiDAR image and say, ‘That’s a cat or a dog or a child.’
The exciting development is that companies like Nvidia, Intel, and others are also providing AI tools that we can use to train these systems to interpret those radar and LiDAR images better than a human could. As I mentioned, these sensors capture a tremendous amount of information which has to be shared over the vehicle network and processed by the on-board computing platforms.
And this is the other area where AI plays an important role. Once you have processed the sensor information you can build a real-time model of the environment around the automated vehicle. We then use AI to help us predict what we think all of the agents (people, cars, bikes, …) in the environment are and will be doing (also referred to as semantic understanding). It’s not enough to know there is a pedestrian by the side of the road, you also need to predict what that pedestrian will do. AI helps us develop this semantic understanding so we can adjust the behavior of the vehicle for a given situation.
Finally, we use AI to optimize the path planning of the vehicle – to solve for the optimal path given all of the surrounding constraints and variables. You can’t make a rule for every situation the car will encounter and AI helps us solve those corner cases. This hybrid approach, using AI based modules within a deterministic framework gives us the best of both worlds: clear generalized rules and policies governing the overall behavior of the vehicle and AI based algorithms to help us solve the most complex corner cases. In order to develop and autonomous driving systems we will need to use all the tools at our disposal and AI will be an important part of the solution.
And that is what we always keep at the top of our mind – arriving safely at the destination. You can be sure our engineers are working tirelessly to provide the safest automated ride out there – and we won’t stop trying to get better – ever.
At the end of the day, several of the reporters thanked me for helping them understand these complicated topics. One even said, “I’ve been writing about AI for ten years and have been thinking about it all wrong.”
For engineers, technology is at the heart of everything we do and even how we think. We break down complex technical issues into the simplest possible elements and then put them back together again. Sometimes just in our own minds – but it helps us understand how things work. Once we understand that, we can come up with ways to make things work better.
And AI is tool we can use to solve problems that can’t be solved using classical techniques. AI has been around for many years; but we now have cost-effective processors allowing us to solve highly complex problems that were virtually impossible to solve before.
The key for AI is training – and training means collecting massive amounts of data that can be used to help train the AI algorithms (neural nets) to perform their task. AI isn’t the right tool for every job, but AI can be used to make our systems much more capable and robust.
And that’s what Delphi is trying to do AI and our AVT. So how do we use AI for our automated vehicles? The answer is, very carefully. We use AI algorithms within what we call a deterministic framework: a set of over-arching policies and rules that govern how the vehicle will operate (for instance, you can set a rule that you will never drive faster than the speed at which the vehicle is able to stop within the free space around the car). Additionally, you don’t need AI to solve for vehicle dynamic behavior or to know what the speed limit is; these are things that can be explicitly defined or modeled. We use AI for functions such as object classification, behavior modification, trajectory optimization, and vehicle to environment socialization. This is where AI can really add value – solving problems where you simply can’t write enough rules to cover all of the situations or corner cases.
So where does all of the data required for training come from? The primary means is through the car itself; its eyes and ears, or its sensors. We call that ‘on-board.’ There are three main types of sensors – vision (cameras), radar and LiDAR. While some companies are using only one or two types of sensors, Delphi believes you need all three types of sensors because none of them are 100 percent perfect in all operating conditions. Together, however, they create redundancies which makes for a much more robust perception system. We believe this strategy is critical to safety, as well as to ensure the car can, and will, operate in any condition. These sensors take in more than 40 Terabytes of data an hour and AI plays a critical role in processing this data - identifying, classifying and predicting what is happening around the vehicle.
And so while we drive… and drive… and drive some more, we continually ask the question, ‘How do we refine the system? How do we make it more adaptable, flexible and more comprehensive?’
The answer lies in part, to the use of AI. But, before I go too deep, too fast, let’s start with the basics. AI is defined as ‘any device that perceives its environment and takes actions that maximize its chance of success at some goal.’ For autonomous vehicles, that means getting safely to a predetermined destination.
What does that all really mean in practical terms – and for our current development state, how do we manage all that data?
The cameras or vision systems were the first to use AI and are currently the heaviest users. Why? A camera alone doesn’t do much. It captures images, but it doesn’t know what those images are or how to react to them. That’s where AI and machine learning come into play.
You have to train the system - and this requires technicians to annotate the images our sensors collect so that they can be used to teach the system - ‘that’s a dog, a cat, a tree, a child, a street sign.’ The more images can be used to train the system, the better the system will perform. This is especially important for object classification where we are attempting to determine what exactly the camera systems are seeing.
As I said earlier, we don’t stop there because we believe you need to use all three sensors, which also include radar and LiDAR. A human can’t easily look at a radar or LiDAR image and say, ‘That’s a cat or a dog or a child.’
The exciting development is that companies like Nvidia, Intel, and others are also providing AI tools that we can use to train these systems to interpret those radar and LiDAR images better than a human could. As I mentioned, these sensors capture a tremendous amount of information which has to be shared over the vehicle network and processed by the on-board computing platforms.
And this is the other area where AI plays an important role. Once you have processed the sensor information you can build a real-time model of the environment around the automated vehicle. We then use AI to help us predict what we think all of the agents (people, cars, bikes, …) in the environment are and will be doing (also referred to as semantic understanding). It’s not enough to know there is a pedestrian by the side of the road, you also need to predict what that pedestrian will do. AI helps us develop this semantic understanding so we can adjust the behavior of the vehicle for a given situation.
Finally, we use AI to optimize the path planning of the vehicle – to solve for the optimal path given all of the surrounding constraints and variables. You can’t make a rule for every situation the car will encounter and AI helps us solve those corner cases. This hybrid approach, using AI based modules within a deterministic framework gives us the best of both worlds: clear generalized rules and policies governing the overall behavior of the vehicle and AI based algorithms to help us solve the most complex corner cases. In order to develop and autonomous driving systems we will need to use all the tools at our disposal and AI will be an important part of the solution.
And that is what we always keep at the top of our mind – arriving safely at the destination. You can be sure our engineers are working tirelessly to provide the safest automated ride out there – and we won’t stop trying to get better – ever.
