Survey: Exploiting Data Redundancy for Optimization of Deep Learning

Abstract

Data redundancy is ubiquitous in the inputs and intermediate results of Deep Neural Networks (DNN). It offers many significant opportunities for improving DNN performance and efficiency and has been explored in a large body of work. These studies have scattered in many venues across several years. The targets they focus on range from images to videos and texts, and the techniques they use to detect and exploit data redundancy also vary in many aspects. There is not yet a systematic examination and summary of the many efforts, making it difficult for researchers to get a comprehensive view of the prior work, the state of the art, differences and shared principles, and the areas and directions yet to explore. This article tries to fill the void. It surveys hundreds of recent papers on the topic, introduces a novel taxonomy to put the various techniques into a single categorization framework, offers a comprehensive description of the main methods used for exploiting data redundancy in improving multiple kinds of DNNs on data, and points out a set of research opportunities for future to explore.

Cloud‐based video streaming services: Trends, challenges, and opportunities

“…The quality of the compressed video is sacrificed in order to remove as much redundancy as possible. Retaining too much redundancy can increase file size and decrease efficiency, while removing too much redundancy can lead to artefacts and other quality problems [19].…”

EG-HRNet: An Efficient High-Resolution Network Using Ghost-Modules for Human Pose Estimation

“…The key to it is to detect the key points of the human body that can effectively represent the human pose. [33] To achieve high performance and accuracy in detecting the key points, high-resolution feature extraction has been emphasized in recent works [9,12,14,16,18,24].…”