An architecture-level analysis on deep learning models for low-impact computations

Abstract

Deep neural networks (DNNs) have made significant achievements in a wide variety of domains. For the deep learning tasks, multiple excellent hardware platforms provide efficient solutions, including graphics processing units (GPUs), central processing units (CPUs), field programmable gate arrays (FPGAs), and application-specific integrated circuit (ASIC). Nonetheless, CPUs outperform other solutions including GPUs in many cases for the inference workload of DNNs with the support of various techniques, such as the high-performance libraries being the basic building blocks for DNNs. Thus, CPUs have been a preferred choice for DNN inference applications, particularly in the low-latency demand scenarios. However, the DNN inference efficiency remains a critical issue, especially when low latency is required under conditions with limited hardware resources, such as embedded systems. At the same time, the hardware features have not been fully exploited for DNNs and there is much room for improvement. To this end, this paper conducts a series of experiments to make a thorough study for the inference workload of prominent state-of-the-art DNN architectures on a single-instruction-multiple-data (SIMD) CPU platform, as well as with widely applicable scopes for multiple hardware platforms. The study goes into depth in DNNs: the CPU kernel-instruction level performance characteristics of DNNs including branches, branch prediction misses, cache misses, etc, and the underlying convolutional computing mechanism at the SIMD level; The thorough layer-wise time consumption details with potential time-cost bottlenecks; And the exhaustive dynamic activation sparsity with exact details on the redundancy of DNNs. The research provides researchers with comprehensive and insightful details, as well as crucial target areas for optimising and improving the efficiency of DNNs at both the hardware and software levels.

Optimization of U-shaped pure transformer medical image segmentation network

“…The Swin Transformer is the basic unit block. For the encoder, the medical image is segmented into non-overlapping patches ( Li et al, 2023 ) of varying sizes by a patch splitting module. In addition, a linear embedding layer maps the raw-valued features to arbitrary dimensions.…”

“…With the development of deep learning, computer vision technology has made immense splash in the field of medical image analysis. Medical image segmentation has become an important branch of medical image analysis ( Chen et al, 2021 ; Li et al, 2022 , 2023 ; Yue et al, 2022 ). Stable and highly accurate medical image segmentation can greatly improve the clinical speed and diagnostic accuracy of doctors.…”

An Ultralightweight Object Detection Network for Empty-Dish Recycling Robots

“…In many real-world applications, object detection must be performed in a timely and power-saving manner with computational resource constraints. Many other vision tasks have built lightweight models using methods, such as weight quantization [16], [17], network compression [18], computationally efficient architecture design [19], [20], [21], and so on. For some vision tasks, lightweight networks aim to achieve the best tradeoff between accuracy and efficiency, showing their superiority by reducing the model size and FLOPs with a little performance drop [22].…”

Feature Map Analysis-Based Dynamic CNN Pruning and the Acceleration on FPGAs

“…This part analyzes the redundancy feature maps, each of which represents feature information essential for the network. However, according to the investigation, the activation function, a common structure in mainstream networks, results in huge sparsity in the feature maps [32]. A high degree of sparsity means a high percentage of zero elements, which is redundant data in the feature map.…”