Distributed Dynamic Map Fusion via Federated Learning for Intelligent Networked Vehicles

Abstract

The technology of dynamic map fusion among networked vehicles has been developed to enlarge sensing ranges and improve sensing accuracies for individual vehicles. This paper proposes a federated learning (FL) based dynamic map fusion framework to achieve high map quality despite unknown numbers of objects in fields of view (FoVs), various sensing and model uncertainties, and missing data labels for online learning. The novelty of this work is threefold: (1) developing a three-stage fusion scheme to predict the number of objects effectively and to fuse multiple local maps with fidelity scores; (2) developing an FL algorithm which fine-tunes feature models (i.e., representation learning networks for feature extraction) distributively by aggregating model parameters; (3) developing a knowledge distillation method to generate FL training labels when data labels are unavailable. The proposed framework is implemented in the Car Learning to Act (CARLA) simulation platform. Extensive experimental results are provided to verify the superior performance and robustness of the developed map fusion and FL schemes. I. INTRODUCTIONThe intelligent networked vehicle system (INVS) is an emerging vehicle-edge-cloud system that accomplishes cooperative perception, map management, planning and maneuvering tasks via vehicle-to-everything (V2X) communication [1]-[3]. Among all the tasks, distributed map management aims to enlarge sensing ranges and improve sensing accuracies for individual vehicles and plays a central role in . While static maps describe stationary objects (e.g., roads, buildings, and trees), dynamic maps emphasize updating information of mobile objects (e.g., pedestrians, cars, and animals) in real time. Fig. 1 shows the architecture of an intelligent networked vehicle system. There are three main steps for dynamic map fusion: (1) local sensing and perception, (2) local map fusion and uploading, and (3) global map fusion and broadcasting.

Edge-Aided Sensor Data Sharing in Vehicular Communication Networks

“…Sensor data sharing is also being considered in the standardization processes of vehicular communication [10], [11]. To acquire a better estimation of the states of a target vehicle, e.g., its position and velocity, based on sensor data sharing, [12]- [14] investigated data fusion methods.…”

Multi-Agent Cooperative Camera-Based Evidential Occupancy Grid Generation

“…In order to make a cooperative system, the question of the data sharing level arises. Several strategies have been investigated in the state of the art [16]. Either raw data, directly provided by sensors can be shared [15], [4] or data in the form of labels where the agents do most of the processing locally before sharing them as in [5], [6], [3].…”

An Overview of Federated Learning at the Edge and Distributed Ledger Technologies for Robotic and Autonomous Systems

“…In [85], cooperative SLAM based on visual-Lidar have been proposed by deploying a federated deep learning algorithm for feature extraction and dynamic map fusion without transferring original images among the robots. In the area of dynamic map fusion, authors in [86] developed a novel fusion scheme among the networked vehicles supported by FL. Superior performance and robustness were then demonstrated in the Car Learning to Act (CARLA) simulation platform.…”