How do you succeed in reliably implementing ever new driver assistance systems and functions for automated driving in the vehicle? Neusoft Reach relies on a validation solution that allows for end-to-end testing from the developers workstation to ECU approval.
Future, intelligently networked, and automated vehicles are highly software-defined. Software and service capabilities are increasingly becoming the decisive competitive factors of the automotive industry. Neusoft Reach already provides an AUTOSAR-compliant software platform for the next generation of vehicles. This software contains a comprehensive advanced driver assistance system/autonomous driving (ADAS/AD) stack supplemented by a driver monitoring system (DMS) for automated driving (Level 2+/Level 4) and a high-precision positioning controller. The goal of Neusoft Reach is to contribute to the safety and efficiency of transportation by providing the world’s most intelligent environment detection and identification technology.
Intelligent Assistance Systems
The ADAS/AD stack is based on multisensor technology and has algorithms that have been machine-trained. The stack meets the regulations for commercial vehicles in China and the requirements of the C-NCAP. It also takes into account the requirements of the Euro-NCAP 2025 roadmap. It currently supports the following functionalities:
- Traffic Jam Assist (TJA),
- Adaptive Cruise Control (ACC),
- Automated Emergency Braking (AEB),
- Forward Collision Warning (FCW),
- Lane Departure Warning (LDW),
- Lane Keeping Assistance (LKA),
- Advanced Parking Assist (APA),
- Around View Monitoring (AVM)
The simulation and validation solution from dSPACE meets our requirements for developing and testing ADAS/AD control units. It helps us achieve our project goals and accelerates the market launch. The overall efficiency in developing ADAS/AD control units has been improved by 70%.
Motivation: Robust and Innovative
The goal of Neusoft Reach is to continuously expand and optimize the ADAS/AD functions. As part of this process, the development team defined performance values for the camera-based functions that can no longer be achieved reliably and efficiently by the development and validation solutions that were previously used. To provide robust functions, early validation of new algorithms for sensor fusion and perception as well as of the overall system must also be easily possible.
Challenge: Availability of Suitable Test Data
A particular challenge during development is to ensure that the right test data is readily available. One aspect is to use sensor data that is as accurate as possible. Another aspect is to have the option to flexibly use a wide range of corner cases, i.e., particularly critical traffic situations. Exact sensor data can be obtained by logging data during test drives. However, real test drives cover relevant corner cases only insufficiently because critical traffic situations are difficult to perform in reality. Moreover, new test drives are required after a sensor has been changed, which makes the procedure rather uneconomical. Therefore, a simulation-based approach is beneficial because it can simulate arbitrary scenarios, including all corner cases, with a high degree of flexibility. However, the artificially generated data must be based on the physical principles of the respective sensor. A highly realistic sensor model is therefore a decisive criterion for the success of the simulation.
Selecting a Validation Solution
For Neusoft Reach, the first task was to evaluate a suitable development and test solution. In addition to realistic sensor models, the intention was to support developers in the early development phases by means of software-in-the-loop (SIL) simulation. Furthermore, there was a requirement to seamlessly reuse the simulations created to validate the real ECU in hardware-in-the-loop (HIL) simulation. So far, no supplier-independent interface standard has been established for processing raw sensor data. Therefore, a test system for raw sensor data generation must support a wide range of sensor interfaces and protocols. This is the only way for Neusoft Reach to make its control units available to a wide range of customers. Last but not least, this kind of development and test solution must be user-friendly and robust. After evaluating various validation systems, we decided to use a software- and hardware-based solution from dSPACE. Decisive factors were the outstandingly realistic sensor simulation, the flexibility in handling the systems when feeding in data, and the consistency between SIL and HIL simulation.
Early SIL Simulation Supports Function Development
By using the dSPACE solution chain, a purely software-based validation solution for perception and fusion algorithms and ADAS/AD functions is available at the developer’s workstation. It is based on the ground truth simulation models of the Automotive Simulation Models (ASM) tool suite and the sensor-realistic camera model from dSPACE Sensor Simulation. The simulation platform is dSPACE VEOS. ASM simulates the movement trajectories of all road users and the traffic infrastructure. With dSPACE Sensor Simulation, a complete 3-D model of the virtual scene is generated to derive a camera image. Via a memory interface, the camera image is subsequently fed into the algorithm to be tested. The algorithm analyzes the image information and provides information that ultimately controls a simulated actuator, for example, a brake simulated in ASM. Naturally, it is possible to use a complete virtual control unit in the simulation which contains, among other things, the developed algorithm. The virtual ECU can be integrated into VEOS parallel to the simulation model, for example, in the form of a Simulink® model, a Functional Mock-up Unit (FMU), or an AUTOSAR-based virtual ECU. To do this, Neusoft Reach uses the V-ECU approach proposed by dSPACE.
With the sensor-realistic camera model, we can reliably simulate highly complex test tasks virtually.
In SIL simulation, the vehicle, environment, and sensor models form a control loop with the virtual ECU.
Creating Realistic Simulation Scenarios
The simulation scenario, i.e., the position and trajectories of all objects of a simulation, is recorded during test drives with real sensors. The data is used to create scenarios, which can now be flexibly modified. For example, the speeds of all road users can be changed individually, distances can be redefined, the sizes of objects can be changed, etc. This enables the developers to determine the performance of an algorithm at an early stage. This also applies to all relevant corner cases. The developers also use manually generated scenarios, such as C-NCAP.
Validating Control Units in HIL Simulation
Whenever a new version of the ADAS/AD software is implemented on the ECU, an ECU test is performed using HIL simulation. To put the control unit into operation, its environment, i.e., the vehicle including the sensor system and the vehicle surroundings, must be fully simulated just like in SIL simulation. In order to fully integrate the ECU hardware and all signal processing stages of the camera into the test, it is necessary to inject the simulated sensor data into the signal chain directly behind the imager chip. For this purpose, the calculated image data must be converted into suitable electrical signals. The Environment Sensor Interface Unit (ESI Unit) is used for this purpose. It is part of a particularly powerful simulation platform: While traffic simulation with ASM is performed on a SCALEXIO system, the model of the camera sensor is run on the Sensor Simulation PC from dSPACE. It is equipped with a graphics card that calculates a 3-D world and sensor characteristics, such as the distortion of the camera lens, in real time based on the ground truth data. The ESI Unit then converts the output of the graphics card into the signals for the control unit and also handles parts of the simulation, such as exposure control. Due to different sensor types involved, the ESI Unit is designed to support a wide range of signal interfaces.
Setup of the validation system for camera control units: The HIL system and the data replay system use the same components.
The multisensor software RTMaps enables us to synchronously replay data from different sensors and vehicle buses.
Data Replay Tests
In addition to SIL and HIL tests, we also perform tests with real data. The real data is used, for example, to simulate errors in the laboratory that oc- curred during test drives. The same test setup is used as in HIL simulation: SCALEXIO, Sensor Simulation PC, and ESI Unit. For data replay, the data is played back from the Sensor Simulation PC with RTMaps. The PC transfers the data both to the ESI unit (camera raw data) and to the SCALEXIO sys-tem (bus communication, for example, CAN). There, the data is buffered and then converted time-synchronously to electrical signals for the control unit. The test setup can therefore be used for open-loop and closed-loop tests without any modifications.
Experience from the Development Project
With the simulation and validation solution from dSPACE, Neusoft Reach has successfully validated and launched several ADAS/AD ECUs. They are used in passenger cars and commercial ve- hicles, such as trucks. One of the validated systems was a 360° panoramic view with five cameras, for all of which a data feed via ESI Unit was implemented. The simple reuse of tests for SIL and HIL procedures proved to be particularly useful: On the one hand, tests created at an early stage of function development can also be used for ECU approval, while on the other hand, function developers can take advantage of comprehensive HIL tests for the SIL procedure. The developed tests can be consistently reused between the simulation platforms VEOS and SCALEXIO. The project has also brought about fault injection as a valuable feature that we use to check the reliability of the systems, for example, if pixel- or line-based color errors or noise occur. These errors can be injected automatically or manually. The flexibility of the ESI Unit with regard to sensor interfaces and protocols, such as Maxim GMSL1 and GMSL2, TI FPD-Link III and MIPI CSI-2, is excellent. This puts us in a position to meet the requirements of different OEMs with one system and to adapt to special requirements of suppliers. To implement the software on the AUTOSAR-based ECUs, we use the production code generator TargetLink. Its powerful AUTOSAR functions simplify the creation of AUTOSAR-compliant software.
Without the ESI Unit from dSPACE, it is almost impossible to efficiently test sensors from different manufacturers. It is a valuable component of our validation solution.
The robustness of the ADAS/AD algorithms is improved by the targeted injection of errors via test software.
Role and Assessment of the dSPACE Solution
The solution from dSPACE, which consists of powerful, highly realistic simulation software, a HIL simulator, a high-performance PC for sensor simulation, and the ESI Unit, plays a key role in the approval of a new ECU. The early and consistent use of the SIL and HIL methods as well as the replay process make it possible to shorten software development cycles and improve the quality of the software. This adds significantly to reducing costs and makes an important contribution to timely market introduction. Compared to previous projects without the dSPACE solution, the overall efficiency has improved by 70%. This means that the investment costs have already been offset to an above-ave-rage extent.
Outlook
Neusoft Reach is working on more assistance functions and is further expanding the scope of the systems for automated driving. We work closely with dSPACE to adapt the validation solution to new requirements. In the area of data recording, we are evaluating how the data logging system AUTERA helps us make our workflows more efficient. The latest advancement of sensor-realistic simulation software from dSPACE is also on our agenda as it promises a highly accurate environment sensor simulation with realistic light and weather effects.
Yan Wei, Neusoft Reach
At a Glance
Validation of camera-based driver assistance systems
Challenge
- Testing camera-based control units without stimulation of the imager chip
- Feeding sensor-realistic raw data into the control unit
Solution
- Using an integrated SIL/HIL simulation platform
- Sensor-realistic simulation of camera data with physically correct models
Benefit
- Easily reproducible tests for software and control unit
- 70% increase in test efficiency
About Neusoft
Neusoft offers a rich portfolio of software-based design services, products, and pre-integrated solutions for industries such as automotive, consumer electronics, mobile devices, IoT, medical, business process outsourcing, finance, security and insurance, information technology outsourcing, and more. Neusoft employs more than 20,000 people worldwide in Asia, Europe, North America, and the Middle East. Neusoft focuses on software technology and covers the areas of navigation, ADAS, HMI, connected car, and infotainment with its products and solutions for the automotive industry. Neusoft serves a large number of automotive OEMs and Tier 1 customers around the world, including North America, Europe, and Asia. Neusoft Reach Automotive Technology Co., Ltd. (‘Neusoft Reach’ for short) is an innovative enterprise focusing on the application of mobile Internet, artificial intelligence, and new energy technologies in the automotive industry.
About the authors:
Yan Wei
Yan Wei is responsible for ADAS at Neusoft Reach Automotive Technology (Shenyang) Co., Ltd. in Shenyang, China.
Long Ning Zhao
Neusoft Reach Automotive Technology (Shenyang) Co., Ltd. in Shenyang, China.
Di Wu
Neusoft Reach Automotive Technology (Shenyang) Co., Ltd. in Shenyang, China.
Xiao Yu Chen
Neusoft Reach Automotive Technology (Shenyang) Co., Ltd. in Shenyang, China.
Ding Nan
Neusoft Reach Automotive Technology (Shenyang) Co., Ltd. in Shenyang, China.
