SmartKai is a digital assistance system that aims to make damage caused by maneuvering in the tight space of a harbor basin a thing of the past. Lidar sensors installed in the harbor capture each ship’s position data, and with the help of the RTMaps software, the data is then prepared for display to the ship’s crew.

SmartKai is an application-oriented research and development project complying with the funding guideline on innovative harbor technology (IHATEC) issued by the German Federal Ministry of Transport and Digital Infrastructure (BMVI), in which several partners are cooperating:

• Niedersachsen Ports GmbH und Co.KG (NPorts) – Germany’s largest port operator and coordinator of the SmartKai project. NPorts makes its harbor infrastructure available for the development of SmartKai. Quays and harbor locks with a high risk of accident are good locations for using the system.
• SICK AG – Is developing a new, more robust lidar sensor for the project, designed specially for a maritime environment. The sensor uses an adapted light wavelength with a longer range.
• Humatects GmbH – Is developing SmartKai’s user interface so that ship crews can easily visualize navigation data using a tablet, augmented reality glasses, or projections.
• OFFIS – Informatics institute at which the harbor’s lidar sensor system for capturing ship positions is being developed.
• eMIR (eMaritime Integrated Reference Platform) – Development platform for maritime applications.

Parking assistants have long been a familiar feature in cars. Now ships will soon have something similar. Maneuvering a ship in the confined space of a harbor basin can be difficult due to adverse currents, bad weather, the increasing size of ships, the growing amount of ship traffic, tight schedules, and so on. The result is repeated accidents that cause damage to both the vessels and the harbor infrastructure – and sometimes personal injuries. The SmartKai project aims to prevent this.

First Prototype in Wilhelmshaven

The first step was to test the system with no disturbing effects, so Niedersachsen Ports (NPorts) installed it at Hannover Quay in Wilhelmshaven, an almost completely closed harbor area with almost no currents or tides. Three 2-D lidar sensors from project partner SICK were the basis for SmartKai. In addition, environmental sensors (for wind data and visibility) and an AIS receiver were installed (AIS = Automatic Identification System, an internationally standardized radio system for exchanging navigation and other vessel data). Two cameras for visual monitoring were mounted at strategic points on the quayside. All the sensors are connected to two industrial PCs (IPCs) in mobile boxes on the quay. These collect, save, and process the measurement values from the sensors.

Processing the Measurement Values with RTMaps

For a safety- and time-critical system like SmartKai, reliability and processing speed are essential. The specific challenge is to assign accurate time data to the captured measurement values and synchronize the processing instances, which are spread over a wide area. So the obvious course was to use the RTMaps software (Real-Time Multisensor Applications), which was created for just such scenarios where users have to collect, time-stamp, synchronize, and replay data from different sensors. To handle the challenges posed by the SmartKai project, there is an RTMaps run-time instance on each of the industrial PCs. In addition, all the RTMaps instances are synchronized with one another to ensure that the system is real-time-capable. Another of the project’s main aims is to store all the incoming data permanently to make it available for further development and evaluation of the system. After over a year in operation, an extensive collection of lidar, camera, and AIS and wind data has been collected. RTMaps is also able to replay this data synchronously.

Wide Range of Scenarios Tested Extensively

In September 2020, the system in Wilhelmshaven underwent its first three-day field test with more than 20 scenarios. These were oriented to the regulations issued by the International Maritime Organization (IMO), as well as to requirements stated by pilots. The research boat Josephine (belonging to OFFIS e.V.) and the survey ship Argus (NPorts) were used for the test. The sensor measurement values were saved via RTMaps and appropriately annotated. Data on the start and end times of the scenarios, plus details of any issues or events such as passing ships, were saved in the RTMaps EventMarker format so that the data could be used for the further development of the system. Following that, the system continued in operation through winter 2020/2021 and summer 2021. Docking maneuvers under harsh weather conditions are of particular interest because they can show the effect of the weather on the lidar measurement values. Throughout the entire test phase, data was continuously captured, time-stamped, and saved via RTMaps.

dSPACE MAGAZINE, PUBLISHED NOVEMBER 2021