Context-Aware Convolutional Neural Network over Distributed System in Collaborative Computing

Choi, Jinhang; Hakimi, Zeinab; Shin, Philip Wootaek; Sampson, Jack; Narayanan, Vijaykrishnan

doi:10.1145/3316781.3317792

Cited by 13 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In (25), the total loss of the internal predictors is shown in the first term that takes into account all possible stopping points. The second term in the VDDIB-SR objective acts as an auxiliary loss, which maintains the informativeness of each encoded feature and makes the training process robust against dynamic activation caused by the attention module.…”

Section: B Vddib-sr Objectivementioning

confidence: 99%

“…• CAFS [25]: The method integrates a context-aware feature selection (CAFS) scheme into the multi-device cooperative edge inference system. Each edge device extracts a taskrelevant feature and determines its importance via entropy-based likelihood estimation.…”

Section: Performance Evaluationmentioning

confidence: 99%

“…After receiving the extracted features, the NN-REG, NN-GBI, and VDDIB methods use vector concatenation for feature aggregation. The CAFS method leverages a weighted pooling layer to fuse the received features, where the weights are determined by the likelihood estimation module [25]. Besides, the eSAFS method performs signification-aware feature selection [4] to identify the most informative feature maps for transmission.…”

Section: ) Neural Network Architecturementioning

confidence: 99%

“…In other words, the redundancy among the dependent sources can be identified and eliminated in transmission. This classic result has recently received renewed attention and led to several works [23], [24], [25] to develop DNN-based distributed feature encoding schemes in task-oriented communication. The authors of [23] developed an optimal distributed quantization system tailored to the classification task, and proposed a greedy search algorithm as part of the deep learning approach to create rectangular quantization regions.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Task-Oriented Communication for Multi-Device Cooperative Edge Inference

Shao¹,

Mao²,

Zhang³

2021

Preprint

View full text Add to dashboard Cite

This paper investigates task-oriented communication for multi-device cooperative edge inference, where a group of distributed low-end edge devices transmit the extracted features of local samples to a powerful edge server for inference. While cooperative edge inference can overcome the limited sensing capability of a single device, it substantially increases the communication overhead and may incur excessive latency. To enable low-latency cooperative inference, we propose a learning-based communication scheme that optimizes local feature extraction and distributed feature encoding in a task-oriented manner, i.e., to remove data redundancy and transmit information that is essential for the downstream inference task rather than reconstructing the data samples at the edge server. Specifically, we leverage an information bottleneck (IB) principle to extract the task-relevant feature at each edge device and adopt a distributed information bottleneck (DIB) framework to formalize a single-letter characterization of the optimal rate-relevance tradeoff for distributed feature encoding. To admit flexible control of the communication overhead, we extend the DIB framework to a distributed deterministic information bottleneck (DDIB) objective that explicitly incorporates the representational costs of the encoded features. As the IB-based objectives are computationally prohibitive for high-dimensional data, we adopt variational approximations to make the optimization problems tractable. To compensate the potential performance loss due to the variational approximations, we also develop a selective retransmission (SR) mechanism to identify the redundancy in the encoded features of multiple edge devices to attain additional communication overhead reduction. Extensive experiments evidence that the proposed task-oriented communication scheme achieves a better rate-relevance tradeoff than baseline methods.

show abstract

Section: B Vddib-sr Objectivementioning

confidence: 99%

Section: Performance Evaluationmentioning

confidence: 99%