Validation Must Be Scalable

Validation Must Be Scalable

The automotive industry is currently trending toward more automation. It won’t be long before we are being taxied from A to B by mobile shuttles or can simply push a button after a hard day’s work to have our car take over the wheel in certain situations. This will allow us to start unwinding even in the car on our way home. Wouldn’t that be wonderful? However, some work still needs to be done to make this dream come true. The technology itself has matured and autonomous driving is already a reality in some areas. Just think of Phoenix, Arizona, where Waymo is already offering citizens autonomous driving services, even without a safety driver. This speaks volumes about the trust Waymo has in its technology.

Trust is a key word here. Whenever people entrust technology with their safety, as is the case with autonomous driving, trust is pivotal. A certain level of trust is needed for new technologies to be accepted and commercially successful. Trust can be built up only if people understand how the technology works and, most importantly, how it was tested. If a technology uses components that are based on artificial intelligence, building trust among the general public becomes particularly challenging. The term “artificial intelligence” alone evokes fear in many. Many people become nervous at the thought of placing their lives in the hands of this inscrutable technology.  

But how do you build up trust? Building up trust is possible only if manufacturers can explain to people that their systems have been thoroughly tested in every conceivable traffic situation. Looking at the following example, it quickly becomes clear that this cannot be achieved in regular test drives with physical prototypes: As early as 2016, RAND Corporation in the US published a study showing that a vehicle would have to autonomously drive 8 billion miles (approximately 12.9 billion kilometers) without a fatal accident to statistically prove that autonomous driving is safer than a human driver because human drivers cause 1.09 fatal accidents per 100 million miles driven. Let’s assume that a manufacturer has a fleet of 100 such autonomous vehicles, which would be quite a fleet seeing as these vehicles are expensive due to the high-quality sensors they contain. If these vehicles were to drive at a speed of 40 mph (approximately 64 km/h), 24 hours a day, 365 days a year, then it would take 228 years for this fleet to achieve 8 billion miles. I think we can all agree that this would not be practical. After all, one would have to start testing all over again if only one line of code or one parameter were to be changed in the vehicle software. There are two main factors that make it possible to validate autonomous vehicles. First, a paradigm shift away from the number of driven kilometers in favor of scenario-based testing. Second, moving the majority of tests to a virtual test environment. Such a software-based test environment makes it possible to scale tests and therefore perform a greater number of tests in a shorter period of time.

In scenario-based testing, certain driving situations are selected in which the driving functions are brought to their limits by special traffic constellations or traffic participants with erratic behavior. These situations can be simulated in a virtual test environment, where they can be used and reused to test the vehicle function without posing a safety hazard to anyone involved. If these scenarios can also be abstracted and made parameterizable, changing the parameter values of a single scenario can generate a plethora of similar scenarios that push the driving function to its limits. This, in turn, means a skyrocketing number of tests that have to be completed to make sure that the function behaves correctly in every situation that could possibly occur in road traffic.

The virtual test environment in which these scenarios are executed and the driving function is tested also has to meet certain criteria so that it can be used for safety-relevant tests of driving functions. First, the models used for the vehicle behavior, such as vehicle dynamics, motor, brakes, etc., have to simulate realistic behavior. One way to ensure this is to complete specific driving maneuvers in a real test drive and then in the virtual simulation environment before comparing the two. This is also the way to validate models that replicate the reaction of other traffic participants to the vehicle under test. Furthermore, the virtual sensors have to be able to perceive the environment and vehicle under test as they would in reality. This is particularly important in later development phases, where environment detection is also tested in the virtual world. Second, the results must be reproducible. This can only be achieved by using deterministic models. The models must be deterministic with short computation times. This is the only way to complete the scenarios as computation time is finite. It is absolutely vital that all models can be simulated together in real time or even faster, especially when it comes to systems that are tested on hardware-in-the-loop test benches, where the final ECU is available as real hardware while the surrounding vehicle, including the vehicle environment, is virtual and the ECU is fed the required signals. If signals do not reach the ECU in the correct time frame, the ECU enters an error state and the test is no longer meaningful.

But high-performance models are also required for software-in-the-loop environments, where both the simulation environment and the function under test are available as software. By scaling this simulation environment up, for example, by using cloud technology, many tests can be executed in parallel, but improved model performance means that more tests can be completed in the same time or the same number of tests can be completed faster. This saves valuable test and development time, which translates into lower costs.

dSPACE has created an entirely new type of simulation and validation software: SIMPHERA. It fulfills all the above requirements because it is based on established dSPACE software, which is considered the industry standard. In addition, by using web and cloud technologies, dSPACE makes it possible to connect top development teams that are distributed across the globe. This creates entirely new synergies that simply would not be available with local validation solutions. The clean and user-friendly SIMPHERA interface also offers the benefits of virtual validation to those who are not simulation experts. For example, function developers can use simulation in early development phases to find errors early in the development process, making the vehicle of tomorrow even safer. The virtually boundless scalability of SIMPHERA lets users run tests on innumerable scenarios in parallel, making it possible to build trust in autonomous vehicles: the trust required to ring in a future that is autonomous, safe, and relaxed.

‍