SERES is a global transportation technology company that develops and manufactures intelligent electric vehicles to create safer, cleaner, and more sustainable communities. We at SERES technologies are focused on delivering improved safety, convenience, and performance to a global market. Our company owns and operates manufacturing, assembly, and research and development facilities across the U.S., China, and Japan.

SilkRides and Our AD Strategy

SilkRides is a business unit of SERES that develops autonomous driving (AD) technology. Our team has an OEM heritage with Silicon Valley roots to provide a cost-competitive and open autonomy solution for carmakers. Established in 2017, SilkRides has demonstrated Level 3 and Level 4 autonomous driving in urban and highway scenarios. 

The SilkRides AD Stack

The team is set to cover the complete technology required to develop and manufacture autonomous vehicles. This includes expertise in perception, planning, and control – not only for development but also for the validation of these safety-related areas as well as over-the-air (OTA) software updates, hardware design, etc. A key component is the domain control unit, which is partly based on artificial intelligence. It evaluates the sensor data and determines the driving strategy. In a first step, it is implemented as a prototype controller, which is gradually developed further for mass production.

The Challenge of AD Validation

To prove that an autonomous driving system is as safe as an average human driver, billions of real-world driving miles for validation might be required. Moreover, each update of the software also has to undergo rigorous testing. Here at SERES, we have limited resources and a tight deadline to accomplish this task, and road testing on a large scale is unrealistic. To speed up our development and validation cycle, we look to establish our verification and validation pipeline with various levels of simulation, where key scenarios and corner cases can be tested for each software release before road tests. 

The SilkRides Testing Pipeline

At SilkRides, each software release goes through multiple levels of testing before it reaches the test vehicle. First, unit tests and module tests are performed to ensure that the new software components behave as intended. Then the new components are integrated with the rest of the decision and planning modules for software-in-the-loop (SIL) testing, where the software pilots a simulated vehicle through relevant test cases to evaluate performance. If the SIL testing results are positive, the new software release will be compiled on the domain controller hardware for hardware-in-the-loop (HIL) testing. The process has a certain data consistency for test artifacts, such as virtual roads. During the SIL and HIL testing process, many issues invol-ving the software and the interaction between software and hardware are observed and fixed, reducing the number of test vehicles and road test mileage required to validate the software releases.

Requirements for HIL Testing

There are a few key requirement that have to be solved by HIL testing for the development of autonomous driving at SilkRides:

• In-house integration of software and hardware: HIL tests are intended to provide a method to test the software on our domain controller as early as possible. Any issues caused by embedded software integration or the real-time operation on the hardware can be identified and remedied before testing in the vehicle.
• Integration testing of third-party sensors: With over-the-air (OTA) radar and camera test benches, the integration of the real sensors in the HIL test setup is possible. Issues caused by the sensor drivers, harness, and the sensor itself can be addressed on the HIL test.
• Safe in-lab testing platform for corner cases: Testing critical corner cases and hardware failures can be dangerous on the road, and impossible to simulate in SIL simulation. The HIL setup lets us to validate these potentially dangerous cases in the safety of the lab. Read more on this in the ’Simulation of Fail Operation Functions‘ section.

Test System Setup

To perform detailed and comprehensive lab testing, we went for a flexible and complete setup that covers the SilkRides AD stack. Together with dSPACE, we defined a system that can handle integration testing of sensors, controllers, and actuators. It consists of a HIL simulator that realistically represents our vehicles. It is further extended by four synchronized sensor test benches, which bring the real radars and cameras into the loop. The ability to use real components also allows us to evaluate the performance of different components at the early stages of development. 

Closed-Loop Testing

The simulated world is then fed into the sensor test benches to stimulate the radar and camera sensors. The sensor signals are the inputs for the device under test, which is our controller running the developed AD software with perception and sensor fusion algorithms. This way, all AD components are integrated and their real behavior is taken into account. Every virtual test drive can be monitored by 3-D animation in real time.

Test Possibilities and Results

The test system, based on the HIL simulator and the test benches, gives us various test possibilities that lead to excellent coverage. They also help perform quick evaluations under identical conditions. The following evaluations and tests were performed and show what SERES has achieved:

Flexible Sensor Integration

The field of autonomous driving is quickly evolving, so investing in assets that can adapt to changes is crucial. Over-the-air radar and camera test benches from dSPACE were selected for their flexibility in adapting to different sensor types and sensor configurations. The over-the-air approach makes it particularly easy to perform plausibility tests. This way the behavior of the sensors (complete control unit including the signal processing software) is evaluated under defined conditions – and especially in borderline cases. The cases in which objects are detected by a sensor or not detected are fundamental for the development of the software that further processes the sensor signals.

Sensor Benchmarking with Radar Chamber

SilkRides relies on automotive Tier 1 suppliers for radar sensors, so accurate benchmarking of the performance between the sensors is one of the key tasks to be considered when narrowing down the supplier. Radar performance tests on the vehicle generally require a large site, and often require the host vehicle and the target vehicle to be moving simultaneously with ground truth devices installed. With the closed-loop radar test bench, many of the tests can be conducted in the lab, providing clear and consistent results. The dSPACE Automotive Radar Test System (DARTS), which is installed in every radar test bench, contributes to this. It generates radar echoes over-the-air with moving antennas so that the characteristics of a sensor can be determined exceptionally accurately.

Scalable for Future Applications

Our 1x camera, 1x front radar, 2x corner radar HIL setup is enough to simulate most of the highway driving functions in our system, since the corner radars can be configured to be on the front, side, or rear of the vehicle. As next steps, additional cameras, radars, lidars, ultrasonic sensors, and GNSS simulators can be included on the SCALEXIO platform. Due to the flexible system setup, these can be either hardware or software extensions. For instance, the system supports the integration of a lidar model from the dSPACE Sensor Simulation tool chain that simulates the complete lidar transmission channel, including the 3-D environment.

Simulation of Fail Operation Functions

With the development of autonomous driving systems of SAE Level 3 or above, many parts of the system must be fail-operational in all of the operational design domains. Such failure conditions can be difficult and sometimes dangerous to replicate in the vehicle. The hand-off between primary function and backup function in case of failure and the safe stop maneuvers in various road conditions can all be simulated in the HIL test.

Automation Enabling Regression Testing

dSPACE tools, such as AutomationDesk, allow us to automatically configure the test benches upon startup, which significantly reduces manual entry steps. Moreover, many of our tests can be automatically executed without user intervention, allowing a fixed set of test cases to be systematically evaluated for each new software release or hardware modification.

Scenario Generation with HD Maps: Bay Bridge

The SilkRides highway driving uses third-party maps to assist in lane keeping, lane change, and routing functions. The integration between the dSPACE ASM and HD map import ensures a high-fidelity simulation environment compared to real-world testing. Here in the San Francisco Bay area, the entrances and the exits of the Bay Bridge have presented many challenging scenarios. We have im- ported the Bay Bridge map into ASM and recreated many of these tricky test cases in our HIL test setup.

Summary and Outlook

The dSPACE test system provides a unique way of integrating sensors and controllers in a virtual vehicle and testing them together. The use of this extra-ordinarily realistic test environment allows us to gain meaningful insights into the performance of the hardware and software components used, even in the early stages of development. This accelerates the development process because groundbreaking decisions can be made at an early stage. Since test cases can be easily reused, regression testing can be performed to reliably verify troubleshooting. New requirements can be met by expanding both the flexible test system and the test libraries. We look forward to bringing new vehicles onto the road that have been validated by a robust and reliable test system.

dSPACE MAGAZINE, PUBLISHED JANUARY 2020