Data Logging in Autonomous Driving Easily Leads to Huge Efforts in Further Processing and Selecting the Right Data

Data logging in autonomous driving is a challenging discipline and represents a significant budget item in the development and testing of autonomous or driver assistance functions. The costs are driven by the complexity of the world on one hand and huge data volumes on the other.

The complexity of the world will not change but the data volumes can be reduced – significantly – using some smart techniques to optimize the logging process.

In a prequel to this blog post, we addressed how to reduce data volume by eliminating redundancy in the data (see the previous blog post here). Because the data is collected for development and validation of functions based on neural networks, we use neural networks as well to determine which data is redundant and which is novel. The approach is very efficient and most importantly, very effective. In most driving situations on the highway, 80% to 90% of data volume can be eliminated.

Today, we describe another method of how to record data with consideration of its future use. The principle is based on triggered recording (unlike continuous recording) that stores only data that meets certain conditions of data relevance. Data relevance is usually related to the operational design domain (ODD) of the development program and description of the environment and situations where the vehicle is driving. In addition, the amount of data already collected for the ODD is considered.

Recording Only the Data of Interest

Triggered recording refers to an approach of data recording when the data is stored (written to a storage device) only when a certain predefined event occurs. The recording can be stopped by another event or by a time-out. Unlike triggered recording, continuous recording stores the data the whole time, until the storage device is full.

Triggers can be internal or external. Internal triggers are in-vehicle events (originating within the vehicle, e.g., emergency braking signal, velocity signal, acceleration signal) and external triggers are determined by particular traffic situations or traffic scenes around the ego vehicle. External triggers can be lighting conditions (e.g., night-time, glare), surrounding road infrastructure (e.g., highway, intersection, traffic sign, toll gates), or the existence and behaviour of the traffic participants.

During road testing, the most interesting data to record is from situations when the system under test does not perform as required. Such situations must be analyzed in detail and the root cause investigated. The required triggers are defined based on driver input, direct ECU output, or indirectly based on internal ECU variables.

Starting the recording exactly when the event occurs might provide an incomplete picture. Imagine that the trigger is set to record data only if a brake signal is detected. If this brake signal is the result of an advanced driver assistance system (ADAS) function that relies on the obstacles detected, it will be difficult to improve the function if it is braking too late. In that case, the situation occurring before the braking signal must be recorded and used for improving the detection part of the function.

Therefore, it is important to have a data sequence preceding the trigger event. To address that, it is common to use circular buffers to implement such ‘pre-trigger’ recording. In that case, data is continuously written to a circular buffer (a specific part of a RAM memory) and if no trigger occurs before the buffer is full, the data from the beginning of the buffer is overwritten. When the trigger occurs, the entire buffer is emptied, and the recording sequence from the buffer is written to a storage device.

Trigger settings are an inherent part and a feature of a data logging system. However, their configuration depends on the project needs derived from the targeted ODD and the state of the development. Autonomously controlling a vehicle in good lighting conditions and sparse traffic is relatively easy to achieve in the early development phase. There is enough data for training and there are many situations to validate it. However, the reliability decreases when considering heavier traffic with various traffic participants and unknown manoeuvres. Similarly, glare or dense road construction can reduce the performance of the system. Therefore, such situations might be in the focus of data collection and road tests while the most common situations can be excluded from recording since their data is already sufficiently represented in a data pool.

Before we move on to requirements and solutions on implementing recording triggers, there is one thing to highlight. Whatever logic is behind the implementation of triggers (decision if the trigger fires or not), the decision must happen in real time. There is no slack and all the data processing for the trigger evaluation, all the comparison logic, and the trigger execution itself must happen in real time.

Later, we will go into more detail on the implementation of triggers with the dSPACE data logging solution based on AUTERA and RTMaps. The flexibility of the tools provides comprehensive means of setting a broad variety of triggers with various complexity covering the entire range of potential trigger events.

Automatic Triggers Based on Bus Signals and ECU State

The most common way to trigger a recording is based on analysis of a particular bus signal or network communication. For example, the recording trigger can be set to record only if the vehicle is moving with velocity above 20 km/h. In such cases, a velocity signal is extracted from a CAN bus and periodically compared to the threshold. Another example is recording in a predefined geographical area only. In that case, the position received from a GNSS receiver is used as a trigger signal.

The prerequisite for such a trigger setting is an ability of the logging system to extract information from buses and networks, most commonly CAN, LIN, FlexRay, and Ethernet. Having a communication matrix and appropriate tools solves the issue.

In E/E architectures for ADAS/AD, smart sensors often provide an object list describing the objects around the vehicle via buses and networks which might be also leveraged as a recording trigger. Situations with traffic participants around are usually more interesting than empty roads. Furthermore, the information about the position of the objects can be taken into consideration for a trigger as well (e.g., if the moving object is within 50 m of the ego vehicle).

AUTERA bus and network interfaces with RTMaps provide an easy way to extract bus signals and use them for recording triggers. As a demo example, we have selected a CAN bus and object list from a front and rear camera. The trigger condition is defined as following: start the recording when a truck is detected less than 30 meters from the ego vehicle (in any direction).

We have used a simulator to generate the CAN signals and AUTERA with RTMaps as a logging system. For the entire demo and details on the implementation you can check the full video in the Learning Center platform.

Leveraging Triggered Recording to Save Cost and Time

Versatile triggers provide a powerful means for data reduction and thus, reduction of costs during the recording process. With limited effort and an increase in the complexity of the logging configuration recording triggers bring significant benefit. To fully leverage the potential of triggers for data recording, a suitable platform is necessary; a platform that offers the flexibility in analyzing data content from various data sources and a platform that provides sufficient computational resources to execute the necessary pre-processing and data analysis in real time. The AUTERA data logger, which can also execute complex algorithms for the precise generation of trigger events, provides a suitable solution for these tasks. It is complemented by the software tools RTMaps and RTag, which support the convenient creation of automated and manual triggering.

This compact in-vehicle solution is easy to use. Contact dSPACE to learn more about how AUTERA and the software tools can optimize the data acquisition process and costs for your specific requirements.