Advancing Applications of Robot Audition Systems: Efficient HARK Deployment with GPU and FPGA Implementations

Abstract

This paper proposes efficient implementations of robot audition systems, specifically focusing on deployments using HARK, an open-source software (OSS) platform designed for robot audition. Although robot audition systems are versatile and suitable for various scenarios, efficiently deploying them can be challenging due to their high computational demands and extensive processing times. For scenarios involving intensive high-dimensional data processing with large-scale microphone arrays, our generalizable GPU-based implementation significantly reduced processing time, enabling real-time Sound Source Localization (SSL) and Sound Source Separation (SSS) using a 60-channel microphone array across two distinct GPU platforms. Specifically, our implementation achieved speedups of 23.3× for SSL and 3.0× for SSS on a high-performance server equipped with an NVIDIA A100 80 GB GPU. Additionally, on the Jetson AGX Orin 32 GB, which represents embedded environments, it achieved speedups of 14.8× for SSL and 1.6× for SSS. For edge computing scenarios, we developed an adaptable FPGA-based implementation of HARK using High-Level Synthesis (HLS) on M-KUBOS, a Multi-Access Edge Computing (MEC) FPGA Multiprocessor System on a Chip (MPSoC) device. Utilizing an eight-channel microphone array, this implementation achieved a 1.2× speedup for SSL and a 1.1× speedup for SSS, along with a 1.1× improvement in overall energy efficiency.