DeepWiVe: Deep-Learning-Aided Wireless Video Transmission

Tung, Tze-Yang; Gündüz, Deniz

doi:10.48550/arxiv.2111.13034

Cited by 6 publications

(9 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, existing end-to-end wireless video transmission results, e.g. [36], all demonstrate worse coding gain compared to H.265 + LDPC in the high SNR region.…”

Section: B Reconstruction Task Resultsmentioning

confidence: 97%

“…The test sequences includes Class A (2560 × 1600), Class B (1920 × 1080), Class C (832 × 480), Class D (416 × 240), Class E (1280 × 720), and UVG (1920 × 1080). During model testing, we set the GOP size as N = 4, which is identical to the configuration of end-to-end wireless video transmission scheme in [36]. As for I-frame coding, we adopt our previous work of image semantic transmission using nonlinear transform source-channel coding [10].…”

Section: Resultsmentioning

confidence: 99%

“…3) Comparison Schemes: Following [36], we compare our DVST with classical video coded transmission schemes in current mainstream wireless communication systems. In particular, we employ the standard video codecs (H.264 [39] and H.265 [40]) for source coding combined with practical LDPC codes [26] or ideal capacity-achieving channel code family for channel coding.…”

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Wireless Deep Video Semantic Transmission

Wang¹,

Dai²,

Liang³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we design a new class of high-efficiency deep joint source-channel coding methods to achieve end-to-end video transmission over wireless channels. The proposed methods exploit nonlinear transform and conditional coding architecture to adaptively extract semantic features across video frames, and transmit semantic feature domain representations over wireless channels via deep joint source-channel coding. Our framework is collected under the name deep video semantic transmission (DVST). In particular, benefiting from the strong temporal prior provided by the feature domain context, the learned nonlinear transform function becomes temporally adaptive, resulting in a richer and more accurate entropy model guiding the transmission of current frame. Accordingly, a novel rate adaptive transmission mechanism is developed to customize deep joint source-channel coding for video sources. It learns to allocate the limited channel bandwidth within and among video frames to maximize the overall transmission performance. The whole DVST design is formulated as an optimization problem whose goal is to minimize the end-to-end transmission rate-distortion performance under perceptual quality metrics or machine vision task performance metrics. Across standard video source test sequences and various communication scenarios, experiments show that our DVST can generally surpass traditional wireless video coded transmission schemes. The proposed DVST framework can well support future semantic communications due to its video content-aware and machine vision task integration abilities.

show abstract

“…However, existing end-to-end wireless video transmission results, e.g. [36], all demonstrate worse coding gain compared to H.265 + LDPC in the high SNR region.…”

Section: B Reconstruction Task Resultsmentioning

confidence: 97%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Wireless Deep Video Semantic Transmission

Wang¹,

Dai²,

Liang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Therefore, semantic communication systems generally adopt advanced joint source channel coding (JSCC) and have manifested the advantages to transmit different types of contents (e.g. image [7]- [12], speech [13], video [14], [15]) in a semantic manner.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Bit Rate Control in Semantic Communication With Incremental Knowledge-Based HARQ

Zhou

Zhao

et al. 2022

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

Semantic communication has witnessed a great progress with the development of natural language processing (NLP) and deep learning (DL). Although existing semantic communication technologies can effectively reduce errors in semantic interpretation, most of these solutions adopt a fixed bit length structure, along with a rigid transmission scheme, which is inefficient and lacks scalability when faced with different meanings and signal-to-noise ratio (SNR) conditions. In this paper, we explore the impact of adaptive bit lengths on semantic coding (SC) under various channel conditions. First, we propose progressive semantic hybrid automatic repeat request (HARQ) schemes that utilize incremental knowledge (IK) to simultaneously reduce the communication cost and semantic error. On top of this, we design a novel semantic encoding solution with multibit length selection. In this fashion, the transmitter employs a policy network to decide the appropriate coding rate, so as to secure the correct information delivery at the cost of minimal bits. Moreover, a specific denoiser is further introduced to reduce the semantic errors encountered in the transmission process according to the semantic characteristics of context. Extensive simulation results have been conducted to verify the effectiveness of the proposed solution.

show abstract

“…[14] and [15] designed semantic communication systems that are capable of multimodal data transmission for tasks, such as visual question answering. For the wireless video transmission, a semantic system has been developed in [16] which exploits reinforcement learning to optimize bandwidth allocation and GoP sizes. For the task-oriented communication, the authors in [17] utilized the information bottleneck principle to find a compact representation for a specific task while preserving the semantic-relevant information.…”

Section: Introductionmentioning

confidence: 99%

Semantic-preserved Communication System for Highly Efficient Speech Transmission

Han¹,

Yang²,

Shi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning (DL) based semantic communication methods have been explored for the efficient transmission of images, text, and speech in recent years. In contrast to traditional wireless communication methods that focus on the transmission of abstract symbols, semantic communication approaches attempt to achieve better transmission efficiency by only sending the semantic-related information of the source data. In this paper, we consider semantic-oriented speech transmission which transmits only the semanticrelevant information over the channel for the speech recognition task, and a compact additional set of semantic-irrelevant information for the speech reconstruction task. We propose a novel end-to-end DLbased transceiver which extracts and encodes the semantic information from the input speech spectrums at the transmitter and outputs the corresponding transcriptions from the decoded semantic information at the receiver. In particular, we employ a soft alignment module and a redundancy removal module to extract only the text-related semantic features while dropping semantically redundant content, greatly reducing the amount of semantic redundancy compared to existing methods. We also propose a semantic correction module to further correct the predicted transcription with semantic knowledge by leveraging a pretrained language model. For the speech to speech transmission, we further include a CTC alignment module that extracts a small number of additional semantic-irrelevant but speech-related information, such as duration, pitch, power and speaker identification of the speech for the better reconstruction of the original speech signals at the receiver. We also introduce a two-stage training scheme which speeds up the training of the proposed DL model. The simulation results confirm that our proposed

show abstract

DeepWiVe: Deep-Learning-Aided Wireless Video Transmission

Cited by 6 publications

References 40 publications

Wireless Deep Video Semantic Transmission

Wireless Deep Video Semantic Transmission

Adaptive Bit Rate Control in Semantic Communication With Incremental Knowledge-Based HARQ

Semantic-preserved Communication System for Highly Efficient Speech Transmission

Contact Info

Product

Resources

About