DiSparse: Disentangled Sparsification for Multitask Model Compression

Sun, Xiu‐Li; Hassani, Ali; Wang, Zhangyang; Huang, Gao

doi:10.1109/cvpr52688.2022.01206

Cited by 11 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then the non-local attention mechanism is applied for feature alignment: where ⊗ refers to dot product operation. In practice, the embedded features have half of the channel compared to the original feature and we adopt a neighborhood attention mechanism 47 , which can be regarded as a more efficient implementation of nonlocal attention in our experiments, achieving similar SR performance.…”

Section: Methodsmentioning

confidence: 99%

Time-lapse Image Super-resolution Neural Network with Reliable Confidence Quantification for Optical Microscopy

Qiao,

Liu,

Wang

et al. 2024

Preprint

View full text Add to dashboard Cite

Single image super-resolution (SISR) neural networks for optical microscopy have shown great capability to directly transform a low-resolution (LR) into its super-resolution (SR) counterpart, enabling low-cost long-term live-cell SR imaging. However, when processing time-lapse data, current SISR models failed to exploit the important temporal dependencies between neighbor frames, inclined to generate temporally inconsistent results. Besides, SISR models are subject to inference uncertainty that is hard to accurately quantify, therefore it is difficult to determine to what extend can we trust the inferred SR images. Here, we first build a large-scale, high-quality dataset for the time-lapse image super-resolution (TISR) task, and conducted a comprehensive evaluation on two essential components, i.e., propagation and alignment mechanisms, of TISR methods. Second, we devised the deformable phase-space alignment (DPA) based TISR neural network (DPA-TISR), which adaptively enhances the cross-frame alignment in the phase domain and outperforms existing state-of-the-art TISR models. Third, we combined the Bayesian training scheme with DPA-TISR, dubbed Bayesian DPA-TISR, and designed an expected calibration error (ECE) minimization framework to obtain a well-calibrated confidence map along with each output SR image, which reliably implicates potential inference errors. We demonstrate that the Bayesian DPA-TISR achieves resolution enhancement by more than 2-fold compared to diffraction limits with high fidelity and temporal consistency, enabling confidence-quantifiable TISR in long-term live-cell SR imaging for various bioprocesses.

show abstract

Section: Methodsmentioning

confidence: 99%

Time-lapse Image Super-resolution Neural Network with Reliable Confidence Quantification for Optical Microscopy

Qiao,

Liu,

Wang

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…However, their efficacy has been largely reliant on substantial computational and storage resources, partly due to the extraction of redundant features by convolutional layers. Various model compression strategies and network designs used to improve network efficiency in the past [23]- [25], including network pruning [26]- [27],weight quantization [28] low-rank decomposition [29], and knowledge distillation [30]- [31], etc. However, these methods are all considered as post-processing steps, and thus their performance is usually bounded by the upper bound of a given initial model.…”

Section: Sc_c2f Modulementioning

confidence: 99%

Improved YOLOv8n-based Helmet Wearing Inspection Measurement Method Study

chen,

jiao,

liu

2024

Preprint

View full text Add to dashboard Cite

This paper proposes the YOLOv8n_H method to address issues regarding parameter redundancy, slow inference speed, and suboptimal detection precision in contemporary helmet-wearing target recognition algorithms. The YOLOv8 C2f module is enhanced with a new SC_Bottleneck structure, incorporating the SCConv module, now termed SC_C2f, to mitigate model complexity and computational costs. Additionally, the original Detect structure is substituted with the PC-Head decoupling head, leading to a significant reduction in parameter count and an enhancement in model efficiency. Moreover, the original Detect structure is replaced by the PC-Head decoupling head, significantly reducing parameter count and enhancing model efficiency. Finally, regression accuracy and convergence speed are boosted by the dynamic non-monotonic focusing mechanism introduced through the WIoU boundary loss function. Experimental results on the expanded SHWD dataset demonstrate a 46.63% reduction in model volume, a 44.19% decrease in parameter count, a 54.88% reduction in computational load, and a maximum accuracy improvement to 93.8% compared to the original YOLOv8 algorithm. In comparison to other algorithms, the model proposed in this paper markedly reduces model size, parameter count, and computational load while ensuring superior detection accuracy.

show abstract

“…Model-level resource-efficient learning techniques, such as pruning, quantization, and ensembling of local and global features [2], [20], [21], effectively reduce memory usage while maintaining performance comparable to that of the original dense networks. Recent studies propose resourceefficient MTL methods targeting three aspects: sharing mechanisms [5], [22], interference reduction [6], and task grouping [23]. Sharing mechanisms enable tasks to share model components, reducing model size and resource usage.…”

Section: B Resource-efficient Learningmentioning

confidence: 99%

“…Some tasks might need resource-intensive deep neural networks, while others can achieve satisfactory results with simpler, shallower models. To address these challenges, various MTL approaches [5], [6] have been proposed. These studies focus on network compression through hard parameter sharing, addressing parameter entanglement between tasks.…”

Section: Introductionmentioning

confidence: 99%

Task-Aware Dynamic Model Optimization for Multi-Task Learning

Choi,

Jin,

Kim

2023

IEEE Access

View full text Add to dashboard Cite

Multi-task learning (MTL) is a field in which a deep neural network simultaneously learns knowledge from multiple tasks. However, achieving resource-efficient MTL remains challenging due to entangled network parameters across tasks and varying task-specific complexity. Existing methods employ network compression techniques while maintaining comparable performance, but they often compress uniformly across all tasks without considering individual complexity. This can lead to suboptimal solutions due to entangled network parameters and memory inefficiency, as the parameters for each task may be insufficient or excessive. To address these challenges, we propose a framework called Dynamic Model Optimization (DMO) that dynamically allocates network parameters to groups based on task-specific complexity. This framework consists of three key steps: measuring task similarity and task difficulty, grouping tasks, and allocating parameters. This process involves the calculation of both weight and loss similarities across tasks and employs sample-wise loss as a measure of task difficulty. Similar tasks are grouped based on similarity, and parameters are allocated with dynamic pruning according to task difficulty within their respective groups. We apply the proposed framework to MTL with various classification datasets. Experimental results demonstrate that the proposed approach achieves high performance while taking fewer network parameters than other MTL methods.

show abstract

DiSparse: Disentangled Sparsification for Multitask Model Compression

Cited by 11 publications

References 26 publications

Time-lapse Image Super-resolution Neural Network with Reliable Confidence Quantification for Optical Microscopy

Time-lapse Image Super-resolution Neural Network with Reliable Confidence Quantification for Optical Microscopy

Improved YOLOv8n-based Helmet Wearing Inspection Measurement Method Study

Task-Aware Dynamic Model Optimization for Multi-Task Learning

Contact Info

Product

Resources

About