A 0.57-GOPS/DSP Object Detection PIM Accelerator on FPGA

Jiao, Bo; Zhang, Jinshan; Xie, Yonghong; Wang, Shunli; Zhu, Haozhe; Kang, Xiaoyang; Dong, Zhiyan; Zhang, Lihua; Chen, Chixiao

doi:10.1145/3394885.3431659

Cited by 11 publications

(8 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Better algorithms [24] [2] are designed to more efficiently use resources, improve model performance, and optimize the deployment in a real-world environment. Also, ultra-low-power AI chips [10] and accelerators [15] have been proposed to support always-on ML capability for an extended period by a battery. However, a joint design of hardware and algorithm [30] [13] is required to squeeze the performance since TinyML delivers ML solutions to constrained devices with limited resources.…”

Section: Related Workmentioning

confidence: 99%

How to Manage Tiny Machine Learning at Scale: An Industrial Perspective

Ren¹,

Anicic²,

Runkler³

2022

Preprint

View full text Add to dashboard Cite

Tiny machine learning (TinyML) has gained widespread popularity where machine learning (ML) is democratized on ubiquitous microcontrollers, processing sensor data everywhere in real-time. To manage TinyML in the industry, where mass deployment happens, we consider the hardware and software constraints, ranging from available onboard sensors and memory size to ML-model architectures and runtime platforms. However, Internet of Things (IoT) devices are typically tailored to specific tasks and are subject to heterogeneity and limited resources. Moreover, TinyML models have been developed with different structures and are often distributed without a clear understanding of their working principles, leading to a fragmented ecosystem. Considering these challenges, we propose a framework using Semantic Web technologies to enable the joint management of TinyML models and IoT devices at scale, from modeling information to discovering possible combinations and benchmarking, and eventually facilitate TinyML component exchange and reuse. We present an ontology (semantic schema) for neural network models aligned with the World Wide Web Consortium (W3C) Thing Description, which semantically describes IoT devices. Furthermore, a Knowledge Graph of 23 publicly available ML models and six IoT devices were used to demonstrate our concept in three case studies, and we shared the code and examples to enhance reproducibility: Github

show abstract

Section: Related Workmentioning

confidence: 99%

How to Manage Tiny Machine Learning at Scale: An Industrial Perspective

Ren¹,

Anicic²,

Runkler³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Much work has explored the lightweight of DNNs, such as network pruning [31], [32], knowledge distillation [33], [34], and quantization [35], [36]. DNNs run with low precision operations during inference provide power and memory advantages over full precision, and it also benefits low-bit-width artificial intelligence chip design [37], [38]. The main idea of quantization is to map full precision floating-point numbers to lower precision (8bit or lower) through a quantizer to significantly reduce the amount of floating-point operations (FLOPs) in matrix multiplication.…”

Section: Total Direct Effectmentioning

confidence: 99%

“…In PIM architecture, the quantized network is pre-loaded into BRAM, and intermediate data accessed from/to the off-chip DRAM access is entirely eliminated during inference. This paper implements a part of the CA-SpaceNet into the PIM accelerator proposed by Jiao et al [37], which is demonstrated in Fig. 5(b).…”

Section: E Network Quantizationmentioning

confidence: 99%

See 1 more Smart Citation

CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Space

Wang¹,

Wang²,

Jiang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reliable and stable 6D pose estimation of uncooperative space objects plays an essential role in on-orbit servicing and debris removal missions. Considering that the pose estimator is sensitive to background interference, this paper proposes a counterfactual analysis framework named CA-SpaceNet to complete robust 6D pose estimation of the space-borne targets under complicated background. Specifically, conventional methods are adopted to extract the features of the whole image in the factual case. In the counterfactual case, a non-existent image without the target but only the background is imagined. Side effect caused by background interference is reduced by counterfactual analysis, which leads to unbiased prediction in final results. In addition, we also carry out low-bit-width quantization for CA-SpaceNet and deploy part of the framework to a Processing-In-Memory (PIM) accelerator on FPGA. Qualitative and quantitative results demonstrate the effectiveness and efficiency of our proposed method. To our best knowledge, this paper applies causal inference and network quantization to the 6D pose estimation of space-borne targets for the first time. The code is available at https://github.com/Shunli-Wang/CA-SpaceNet.

show abstract

“…It is about the proliferation of hardware, progress on algorithms, emerging ecosystem, and transformative applications. Ultra-low-power devices have been designed for always-on applications [7] [11]. Various algorithms have been proposed to fully exploit ML models on the devices without compromising performance [22] [17].…”

Section: Related Workmentioning

confidence: 99%

SeLoC-ML: Semantic Low-Code Engineering for Machine Learning Applications in Industrial IoT

Ren¹,

Dorofeev²,

Anicic³

et al. 2022

Preprint

View full text Add to dashboard Cite

Internet of Things (IoT) is transforming the industry by bridging the gap between Information Technology (IT) and Operational Technology (OT). Machines are being integrated with connected sensors and managed by intelligent analytics applications, accelerating digital transformation and business operations. Bringing Machine Learning (ML) to industrial devices is an advancement aiming to promote the convergence of IT and OT. However, developing an ML application in Industrial IoT (IIoT) presents various challenges, including hardware heterogeneity, non-standardized representations of ML models, device and ML model compatibility issues, and slow application development. Successful deployment in this area requires a deep understanding of hardware, algorithms, software tools, and applications. Therefore, this paper presents a framework called Semantic Low-Code Engineering for ML Applications (SeLoC-ML), built on a low-code platform to support the rapid development of ML applications in IIoT by leveraging Semantic Web technologies. SeLoC-ML enables non-experts to easily model, discover, reuse, and matchmake ML models and devices at scale. The project code can be automatically generated for deployment on hardware based on the matching results. Developers can benefit from semantic application templates, called recipes, to fast prototype end-user applications. The evaluations confirm an engineering effort reduction by a factor of at least three compared to traditional approaches on an industrial ML classification case study, showing the efficiency and usefulness of SeLoC-ML. We share the code and welcome any contributions 4 .

show abstract

A 0.57-GOPS/DSP Object Detection PIM Accelerator on FPGA

Cited by 11 publications

References 1 publication

How to Manage Tiny Machine Learning at Scale: An Industrial Perspective

How to Manage Tiny Machine Learning at Scale: An Industrial Perspective

CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Space

SeLoC-ML: Semantic Low-Code Engineering for Machine Learning Applications in Industrial IoT

Contact Info

Product

Resources

About