Today’s autonomous vehicle innovations remind me a lot of an iconic Calvin and Hobbes comic strip, where Calvin claims he is the “culmination of creation.” With all due respect to Calvin, I believe the car is now the true culmination of creation.
Think about it… the connected and autonomous car is not only using a lot of new technology, but is in fact pushing the boundaries of research, investments and innovation in these new technologies. So, when I started making a list of those new technologies, instead of making a traditional Top Ten list, I dialed this one up to the Top Eleven Boundary Pushers. I’m sure I missed a few, so would welcome input.
• IoT – Cars are embedded with hundreds of sensors, converting the physical world around it to digital signals. Imagine complex sensors, such as radars, video cameras and LiDAR, being used to provide the car with 720 degrees (not just horizontal, but vertical as well) of visibility to enable the task of autonomous driving. This makes the car possibly the most IoT enabled product in the market with possibly the most complex and diverse set of sensors. While there are lots of IoT solutions and platforms in other industries, managing a complex and mobile IoT product such as an automobile will require some significant enhancements in capabilities.
• Massive data management – Terabytes of data being generated from systems, sensors and users of the car, all while being converted at high speeds… think multi-gigabytes per hour high speed. The car is fast becoming the largest source of data that mankind ever has seen and it requires new innovations in data management, databases, streaming analytics, security, compression, distributed compute and bandwidth management to name a few.
• Edge analytics – Imagine all these embedded sensors pushing out gigabytes of data every second which the car then uses to navigate precisely; making thousands of decisions. This decision-making requires significant computing ability to fuse and combine the sensor stream at the edge of the network, i.e. in the car itself. We cannot afford to send vast amounts of data to the cloud and wait for decisions to come back – this delay is not acceptable. Analytics will be led from the edge.
• Cloud – While the edge will remain critical for real time analytics and decision-making, a lot of data will still flow back and forth from cloud. Thus, a powerful cloud is critical for managing data, large number of apps in analytics, operations, software updates, high definition mapping, scheduling, payments, visualization, city management and diagnostics to ensure an optimal lifecycle experience of the car.
• Computer vision – Vision is a fundamental technology for sensing the context around a car. Think about analyzing real time videos to understand traffic signals, pedestrians, road obstacles and road signs, and the technology underpinnings around object detection, modelling, path prediction that go into making this work well. The amount of data in video streams, speed of the car and rapid response time constraints all make this a particularly challenging problem. Not to mention the inside-the-car analysis using video around driver behavior, distraction, logging and event correlation. The combined context both internal and external video streams can provide are valuable to a safe and comfortable journey.
• Security – Think back to a 1950’s VW Beetle which had very little software and almost no connectivity. Compared to that, today we have hundreds of processors, microcontrollers, and complex networks shuttling massive quantities of data and control signals back and forth, both inside and outside the vehicle. The complex nature of a moving system with integrated, diverse sub-systems by multiple suppliers requires serious innovations in security and safety protocols. The scale, mobility and diverse conditions in which cars will operate, plus the additional requirements of life and human safety add to this challenge. This then means technologies, such as Blockchain, over-the-air (OTA) updates, hardware roots of trust, malware, security/behavior analytics, identity/certificate management, network and system isolation, and redundancy/safety mechanisms, are just a few that have to be innovated to provide security that works for the new automobile.
• Connectivity – The car is becoming a hyper-connected system of systems. This includes the internal high-speed data and control networks, external facing cellular, 5G, short-range DSRC or V2X, and also the standard set of wireless protocols such as WiFI and BT. In fact, not only are a large number of connectivity methods being used, they also are being integrated along with complex switching, routing and security infrastructure. Managing this connectivity future will require a rethink of both internal and external networking.
• Software – Complex software enables the planning, policy, navigation, safety, and algorithms, as well as the diverse sub-systems and microcontrollers that enable precise control of this new breed of automobile. It is time to think of the modern car not as a mode of transportation alone, but as a software-defined computer on wheels. Automobiles will potentially have over 100M lines of code compared to “smaller systems,” like Facebook (60M), Microsoft Office (45M) and the Android system (15M).
With the size and scale of software usage in cars, innovations in languages, security, performance, real-time capability, software releases and updating will be required.
• Energy – There is arguably more innovation happening in the energy field due to the automobile than from anything else. One only has to look at advances in electric drives, battery technology, induction charging, power regeneration, fuel cells, etc. – all being driven by new automotive needs. With global warming and emissions emerging as a big challenge for cities, newer and more efficient ways to move people and goods are paramount. It is exciting to see how the boundaries are being pushed here in the energy industry.
• Mobile – Mobility and the advent of mobile devices has been a huge step forward in communications, social interactions, access to information and cellular technology. Beyond the smartphone, the advanced mobility use cases are all happening on the automobile such as requirements for massive connectivity at high speeds, streaming content to multiple devices, dynamic mapping based on terrain, user interface innovations around voice/gesture, and new applications built for mobile to name a few. Mobility around automobile scenarios will take a giant step forward from the smartphone.move people and goods are paramount.
• AI – And finally, number 11… To top it off, every innovation in AI and deep learning are being applied to scenarios involving autonomous driving, safety, driver behavior and vehicle diagnostics. Specialized chipsets are being developed for the automobile to take advantage of this and new companies are emerging with an AI view of the world. This list does not even touch on the major technologies in cars to help it move! Think of the innovative software that allows the engine cylinders to fire in perfect harmony, emissions control to reduce pollutants from fuel engines, traction control elements, aerodynamics and hundreds of other cool envelope pushing technologies in our cars.
It’s an amazing time to be in the automotive industry. And like Calvin, we will be fortunate to witness its culmination of creation.
Think about it… the connected and autonomous car is not only using a lot of new technology, but is in fact pushing the boundaries of research, investments and innovation in these new technologies. So, when I started making a list of those new technologies, instead of making a traditional Top Ten list, I dialed this one up to the Top Eleven Boundary Pushers. I’m sure I missed a few, so would welcome input.
• IoT – Cars are embedded with hundreds of sensors, converting the physical world around it to digital signals. Imagine complex sensors, such as radars, video cameras and LiDAR, being used to provide the car with 720 degrees (not just horizontal, but vertical as well) of visibility to enable the task of autonomous driving. This makes the car possibly the most IoT enabled product in the market with possibly the most complex and diverse set of sensors. While there are lots of IoT solutions and platforms in other industries, managing a complex and mobile IoT product such as an automobile will require some significant enhancements in capabilities.
• Massive data management – Terabytes of data being generated from systems, sensors and users of the car, all while being converted at high speeds… think multi-gigabytes per hour high speed. The car is fast becoming the largest source of data that mankind ever has seen and it requires new innovations in data management, databases, streaming analytics, security, compression, distributed compute and bandwidth management to name a few.
• Edge analytics – Imagine all these embedded sensors pushing out gigabytes of data every second which the car then uses to navigate precisely; making thousands of decisions. This decision-making requires significant computing ability to fuse and combine the sensor stream at the edge of the network, i.e. in the car itself. We cannot afford to send vast amounts of data to the cloud and wait for decisions to come back – this delay is not acceptable. Analytics will be led from the edge.
• Cloud – While the edge will remain critical for real time analytics and decision-making, a lot of data will still flow back and forth from cloud. Thus, a powerful cloud is critical for managing data, large number of apps in analytics, operations, software updates, high definition mapping, scheduling, payments, visualization, city management and diagnostics to ensure an optimal lifecycle experience of the car.
• Computer vision – Vision is a fundamental technology for sensing the context around a car. Think about analyzing real time videos to understand traffic signals, pedestrians, road obstacles and road signs, and the technology underpinnings around object detection, modelling, path prediction that go into making this work well. The amount of data in video streams, speed of the car and rapid response time constraints all make this a particularly challenging problem. Not to mention the inside-the-car analysis using video around driver behavior, distraction, logging and event correlation. The combined context both internal and external video streams can provide are valuable to a safe and comfortable journey.
• Security – Think back to a 1950’s VW Beetle which had very little software and almost no connectivity. Compared to that, today we have hundreds of processors, microcontrollers, and complex networks shuttling massive quantities of data and control signals back and forth, both inside and outside the vehicle. The complex nature of a moving system with integrated, diverse sub-systems by multiple suppliers requires serious innovations in security and safety protocols. The scale, mobility and diverse conditions in which cars will operate, plus the additional requirements of life and human safety add to this challenge. This then means technologies, such as Blockchain, over-the-air (OTA) updates, hardware roots of trust, malware, security/behavior analytics, identity/certificate management, network and system isolation, and redundancy/safety mechanisms, are just a few that have to be innovated to provide security that works for the new automobile.
• Connectivity – The car is becoming a hyper-connected system of systems. This includes the internal high-speed data and control networks, external facing cellular, 5G, short-range DSRC or V2X, and also the standard set of wireless protocols such as WiFI and BT. In fact, not only are a large number of connectivity methods being used, they also are being integrated along with complex switching, routing and security infrastructure. Managing this connectivity future will require a rethink of both internal and external networking.
• Software – Complex software enables the planning, policy, navigation, safety, and algorithms, as well as the diverse sub-systems and microcontrollers that enable precise control of this new breed of automobile. It is time to think of the modern car not as a mode of transportation alone, but as a software-defined computer on wheels. Automobiles will potentially have over 100M lines of code compared to “smaller systems,” like Facebook (60M), Microsoft Office (45M) and the Android system (15M).
With the size and scale of software usage in cars, innovations in languages, security, performance, real-time capability, software releases and updating will be required.
• Energy – There is arguably more innovation happening in the energy field due to the automobile than from anything else. One only has to look at advances in electric drives, battery technology, induction charging, power regeneration, fuel cells, etc. – all being driven by new automotive needs. With global warming and emissions emerging as a big challenge for cities, newer and more efficient ways to move people and goods are paramount. It is exciting to see how the boundaries are being pushed here in the energy industry.
• Mobile – Mobility and the advent of mobile devices has been a huge step forward in communications, social interactions, access to information and cellular technology. Beyond the smartphone, the advanced mobility use cases are all happening on the automobile such as requirements for massive connectivity at high speeds, streaming content to multiple devices, dynamic mapping based on terrain, user interface innovations around voice/gesture, and new applications built for mobile to name a few. Mobility around automobile scenarios will take a giant step forward from the smartphone.move people and goods are paramount.
• AI – And finally, number 11… To top it off, every innovation in AI and deep learning are being applied to scenarios involving autonomous driving, safety, driver behavior and vehicle diagnostics. Specialized chipsets are being developed for the automobile to take advantage of this and new companies are emerging with an AI view of the world. This list does not even touch on the major technologies in cars to help it move! Think of the innovative software that allows the engine cylinders to fire in perfect harmony, emissions control to reduce pollutants from fuel engines, traction control elements, aerodynamics and hundreds of other cool envelope pushing technologies in our cars.
It’s an amazing time to be in the automotive industry. And like Calvin, we will be fortunate to witness its culmination of creation.