SoftBank Corp. Develops
“Remote Autonomous Driving Support System”
on AITRAS Edge AI Server

Overview of remote autonomous driving support system

The remote autonomous driving support system developed by SoftBank transmits real-time footage from a front-facing camera installed in an autonomous vehicle to the edge AI server of AITRAS via a 5G network. The perception AI, running on an NVIDIA-accelerated computing platform equipped with a GPU (Graphic Processing Unit) inside an edge AI server, instantly recognizes obstacles and road surface conditions based on the transmitted footage. The results are then sent back to the autonomous vehicle, enabling real-time support for autonomous driving. This AI processing utilizes NVIDIA DeepStream SDK, incorporating complex processing tasks such as neural networks to enable real-time video analysis and tracking. By performing computationally intensive AI processing on the edge AI server and transmitting only the necessary driving-related information to the autonomous vehicle, the system enables autonomous driving without requiring extensive computational resources on the vehicle itself. Currently, the system is designed to support specific driving scenarios, but in the future, aims to provide support across a wide range of driving conditions.

Field trial example

As one of the use cases being tested in the field trial at SFC, an evaluation was carried out to determine whether the remote autonomous driving support system running on the edge AI server could enable autonomous driving equivalent to a standard autonomous driving system in situations where a malfunction occurs in the autonomous driving system while navigating a curve, which would prevent safe driving continuity. In this scenario, certain functions of the autonomous driving system were intentionally disabled while the vehicle was navigating a curve, and instead, support was provided by the remote autonomous driving support system. The perception AI analyzed the surroundings based on camera footage from the autonomous vehicle and transmitted the information to it. The autonomous driving system, operating in risk-minimization control mode, planned the driving path based on the perception results. The test examined whether the vehicle could identify a safe space and come to a stop, instead of stopping immediately, when visibility was obstructed on the curve or when obstacles were detected.

In this case, if lane recognition or obstacle detection is not properly conducted, there is a risk of lane departure or collision with obstacles. Therefore, it is necessary to recognize the drivable area and obstacle positions in real time and, based on the recognition results, determine an appropriate path and provide corresponding control commands. In this field trial, using the remote autonomous driving support system operating on the edge AI server of AITRAS, it was confirmed that autonomous driving was possible even when obstacles were present while navigating a curve.

Integration with traffic understanding multimodal AI

By integrating the remote autonomous driving support system running on the edge AI server of AITRAS with the traffic understanding multimodal AI, autonomous vehicles can continue driving smoothly even when facing complex and unpredictable situations that the autonomous driving system or perception AI alone cannot handle. The traffic understanding multimodal AI is built upon a general-purpose AI foundation model and has been trained with Japanese traffic knowledge, including traffic manuals and regulations, as well as common driving scenarios and risk factors in unpredictable driving situations, along with appropriate countermeasures. This AI analyzes traffic conditions based on driving footage and other data transmitted from the autonomous vehicle and can verbalize the associated risks and appropriate responses in real time. As a result, even if an autonomous vehicle makes an incorrect decision or exhibits abnormal behavior, the system enables safe driving through remote support.

The field trial conducted at SFC included an evaluation scenario of “driving in situations where there are obstacles at a crosswalk.” In this test, the traffic understanding multimodal AI analyzed traffic risks and generated a stop instruction. The remote autonomous driving support system transmitted the results to the autonomous vehicle to evaluate whether it could come to a safe stop. As a result of the evaluation, it was confirmed that operating the system on the edge AI server of AITRAS enabled low-latency transmission of stop instructions to the autonomous vehicle. It was also confirmed that, compared to cases where the edge AI server of AITRAS was not used, the vehicle could stop safely in front of the obstacle.