Fuzzy Logic-Based Adaptive Point Cloud Video Streaming

Liu, Zhi; Li, Jie; Chen, Xianfu; Wu, Celimuge; Ishihara, Susumu; Ji, Yusheng; Li, Jie

doi:10.1109/ojcs.2020.3006205

Cited by 23 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Han et al [10] propose ViVo using a similar approach and employ machine learning models to predict viewport movement. Liu et al [28] follow a similar approach, they include an uncompressed base layer and use fuzzy logic based quality selection. This type of approach using use the leaf nodes of the octree as an enhancement layer is currently not suitable for realtime systems as it adds an extra surface orientation estimation step that introduces additional delays in the pipeline.…”

Section: Compression Standardsmentioning

confidence: 99%

Evaluating the Impact of Tiled User-Adaptive Real-Time Point Cloud Streaming on VR Remote Communication

Subramanyam¹,

Viola²,

Jansen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Remote communication has rapidly become a part of everyday life in both professional and personal contexts. However, popular video conferencing applications present limitations in terms of quality of communication, immersion and social meaning. VR remote communication applications offer a greater sense of co-presence and mutual sensing of emotions between remote users. Previous research on these applications has shown that realistic point cloud user reconstructions offer better immersion and communication as compared to synthetic user avatars. However, photorealistic point clouds require a large volume of data per frame and are challenging to transmit over bandwidth-limited networks. Recent research has demonstrated significant improvements to perceived quality by optimizing the usage of bandwidth based on the position and orientation of the user's viewport with user-adaptive streaming. In this work, we developed a real-time VR communication application with an adaptation engine that features tiled user-adaptive streaming based on user behaviour. The application also supports traditional network adaptive streaming. The contribution of this work is to evaluate the impact of tiled user-adaptive streaming on quality of communication, visual quality, system performance and task completion in a functional live VR remote communication system. We perform a subjective evaluation with 33 users to compare the different streaming conditions with a neck exercise training task. As a baseline, we use uncompressed streaming requiring ca. 300Mbps and our solution achieves similar visual quality with tiled adaptive streaming at 14Mbps. We also demonstrate statistically significant gains to the quality of interaction and improvements to system performance and CPU consumption with tiled adaptive streaming as compared to the more traditional network adaptive streaming.

show abstract

Section: Compression Standardsmentioning

confidence: 99%

Evaluating the Impact of Tiled User-Adaptive Real-Time Point Cloud Streaming on VR Remote Communication

Subramanyam¹,

Viola²,

Jansen³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, Li et al considered the uncompressed tiles and optimize the user's quality of experience (QoE) by selecting the proper quality levels under the communication and computation constraints [13]. Liu et al proposed a fuzzy logic solution to select the quality level for each tile based on the future bandwidth, user FoV, and the available computation capability [14]. However, these schemes require predefined models or rules and thus cannot adapt to the dynamic network conditions.…”

Section: Introductionmentioning

confidence: 99%

FRAS: Federated Reinforcement Learning empowered Adaptive Point Cloud Video Streaming

Gao¹,

Zhou²,

Li³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Point cloud video transmission is challenging due to high encoding/decoding complexity, high video bitrate, and low latency requirement. Consequently, conventional adaptive streaming methodologies often find themselves unsatisfactory to meet the requirements in threefold: 1) current algorithms reuse existing quality of experience (QoE) definitions while overlooking the unique features of point cloud video thus failing to provide optimal user experience, 2) most deep learning approaches require long-span data collections to learn sufficiently varied network conditions and result in long training period and capacity occupation, 3) cloud training approaches pose privacy risks caused by leakage of user reported service usage and networking conditions.To overcome the limitations, we present FRAS, the first federated reinforcement learning framework, to the best of our knowledge, for adaptive point cloud video streaming. We define a new QoE model which takes the unique features of point cloud video into account. Each client uses reinforcement learning (RL) to train encoding rate selection with the objective of optimizing the user's QoE under multiple constraints. Then, a federated learning framework is integrated with the RL algorithm to enhance training performance with privacy preservation. Extensive simulations using real point cloud videos and network traces reveal the superiority of the proposed scheme over baseline schemes. We also implement a prototype that demonstrates the performance of FRAS via real-world tests.

show abstract

“…Academia and industry (such as Fraunhofer Institute for Telecommunications) have also spent great efforts on point cloud video streaming solutions. These recent point cloud video streaming schemes [5][6][7][8][9] are based on the VR video streaming approach [1]. In Fig.…”

Section: Introductionmentioning

confidence: 99%

Point Cloud Video Streaming: Challenges and Solutions

Liu

Chen

et al. 2021

IEEE Network

Self Cite

View full text Add to dashboard Cite

Volumetric video (or hologram video), the medium for representing natural content in VR/AR/MR, is presumably the next generation of video technology and a typical use case for 5G and beyond wireless communications. To realize volumetric video applications, efficient volumetric video streaming is in critical demand. This article responds to the challenges of and propose solutions to wireless transmission systems of point cloud video, which is the most popular and favored way to represent volumetric media and significantly differs from the other types of videos. In particular, we first introduce point cloud video technology and its applications, and then discuss the challenges of and solutions to point cloud video streaming, including encoding, tiling, viewing angle prediction, decoding, quality assessment and transmission optimization. Furthermore, we explain a prototype of MPEG DASH-based point cloud video streaming system as a preliminary study, along with more simulation results to verify its performance. Finally, we identify future research directions for providing high-quality point cloud video streaming.

show abstract

Fuzzy Logic-Based Adaptive Point Cloud Video Streaming

Cited by 23 publications

References 28 publications

Evaluating the Impact of Tiled User-Adaptive Real-Time Point Cloud Streaming on VR Remote Communication

Evaluating the Impact of Tiled User-Adaptive Real-Time Point Cloud Streaming on VR Remote Communication

FRAS: Federated Reinforcement Learning empowered Adaptive Point Cloud Video Streaming

Point Cloud Video Streaming: Challenges and Solutions

Contact Info

Product

Resources

About