BoB: Bandwidth Prediction for Real-Time Communications Using Heuristic and Reinforcement Learning

Abstract

Bandwidth prediction is critical in any Real-time Communication (RTC) service or application. This component decides how much media data can be sent in real time. Subsequently, the video and audio encoder dynamically adapts the bitrate to achieve the best quality without congesting the network and causing packets to be lost or delayed. To date, several RTC services have deployed the heuristic-based Google Congestion Control (GCC), which performs well under certain circumstances and falls short in some others. In this paper, we leverage the advancements in reinforcement learning and propose BoB (Bang-on-Bandwidth) -a hybrid bandwidth predictor for RTC. At the beginning of the RTC session, BoB uses a heuristic-based approach. It then switches to a learning-based approach. BoB predicts the available bandwidth accurately and improves bandwidth utilization under diverse network conditions compared to the two winning solutions of the ACM MMSys'21 grand challenge on bandwidth estimation in RTC. An opensource implementation of BoB is publicly available for further testing and research.

Toward Fair and Efficient Congestion Control: Machine Learning Aided Congestion Control (MLACC)

“…The advancements in ML and artificial intelligence (AI) inspired the research community to utilize these technologies in congestion control such as [2,8,13,24]. The SOTA ML-based CC is Aurora, which depends completely on a deep reinforcement learning (DRL) model.…”

“…It has been demonstrated that Aurora outperforms one of the most popular TCP variants, CUBIC, while being on par with BBR. Another example of ML-based CC is presented in [2], which is a hybrid bandwidth predictor for RTC. It utilizes an initial heuristicbased approach, then switches to a full RL-based model that attempts to learn from network statistics such as receiving rate, packet loss, and packet delay.…”

“…Compared to SOTA CC techniques such as [2,13], our proposed technique does not depend completely on ML to avoid the ML problems such as the lack of generalization in the offline trained models as well as exploration problems of online DRL-based models. Instead, our proposed technique integrates heuristics with ML, where the ML model assists the heuristic by estimating the available ratio of the bottleneck bandwidth.…”