No abstract

Pre-trained large-scale language models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks. However, the limited weight storage and computational speed on hardware platforms have impeded the popularity of pre-trained models, especially in the era of edge computing. In this work, we propose an efficient transformer-based large-scale language representation using hardware-friendly block structure pruning. We incorporate the reweighted group Lasso into block-structured pruning for optimization. Besides the significantly reduced weight storage and computation, the proposed approach achieves high compression rates. Experimental results on different models (BERT, RoBERTa, and DistilBERT) on the General Language Understanding Evaluation (GLUE) benchmark tasks show that we achieve up to 5.0× with zero or minor accuracy degradation on certain task(s). Our proposed method is also orthogonal to existing compact pre-trained language models such as DistilBERT using knowledge distillation, since a further 1.79× average compression rate can be achieved on top of DistilBERT with zero or minor accuracy degradation. It is suitable to deploy the final compressed model on resource-constrained edge devices. We share the related codes and models at: https://bi t.ly/3cvs2N2

No abstract

Image segmentation plays an important role in multimodality imaging, especially in fusion structural images offered by CT, MRI with functional images collected by optical technologies, or other novel imaging technologies. In addition, image segmentation also provides detailed structural description for quantitative visualization of treating light distribution in the human body when incorporated with 3D light transport simulation methods. Here, we first use some preprocessing methods such as wavelet denoising to extract the accurate contours of different tissues such as skull, cerebrospinal fluid (CSF), grey matter (GM), and white matter (WM) on 5 MRI head image datasets. We then realize automatic image segmentation with deep learning by using convolutional neural network. We also introduce parallel computing. Such approaches greatly reduced the processing time compared to manual and semiautomatic segmentation and are of great importance in improving the speed and accuracy as more and more samples are being learned. e segmented data of grey and white matter are counted by computer in volume, which indicates the potential of this segmentation technology in diagnosing cerebral atrophy quantitatively. We demonstrate the great potential of such image processing and deep learning-combined automatic tissue image segmentation in neurology medicine.