A Generalization Performance Study Using Deep Learning Networks in Embedded Systems

Gorospe, Joseba; Mulero, Ruben; Arbelaitz, Olatz; Muguerza, Javier; Antón, Miguel

doi:10.3390/s21041031

Cited by 23 publications

(13 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The high performance of CNNs is essential for on-device intelligence systems, which allows for solving computer vision problems directly on a mobile device without the transmission of information to an external server and thus solve them faster, as well as in a more energy-efficient and secure way [10].…”

Section: Introductionmentioning

confidence: 99%

Fast matrix multiplication for binary and ternary CNNs on ARM CPU

Trusov¹,

Limonova²,

Nikolaev³

et al. 2022

Preprint

View full text Add to dashboard Cite

Low-bit quantized neural networks (QNNs) are of great interest in practical applications because they significantly reduce the consumption of both memory and computational resources. Binary neural networks (BNNs) are memory and computationally efficient as they require only one bit per weight and activation and can be computed using Boolean logic and bit count operations. QNNs with ternary weights and activations (TNNs) and binary weights and ternary activations (TBNs) aim to improve recognition quality compared to BNNs while preserving low bit-width. However, their efficient implementation is usually considered on ASICs and FPGAs, limiting their applicability in real-life tasks. At the same time, one of the areas where efficient recognition is most in demand is recognition on mobile devices using their CPUs. However, there are no known fast implementations of TBNs and TNN, only the daBNN library for BNNs inference. In this paper, we propose novel fast algorithms of ternary, ternary-binary, and binary matrix multiplication for mobile devices with ARM architecture. In our algorithms, ternary weights are represented using 2-bit encoding and binary -using one bit. It allows us to replace matrix multiplication with Boolean logic operations that can be computed on 128bits simultaneously, using ARM NEON SIMD extension. The matrix multiplication results are accumulated in 16-bit integer registers. We also use special reordering of values in left and right matrices. All that allows us to efficiently compute a matrix product while minimizing the number of loads and stores compared to the algorithm from daBNN. Our algorithms can be used to implement inference of convolutional and fully connected layers of TNNs, TBNs, and BNNs. We evaluate them experimentally on ARM Cortex-A73 CPU and compare their inference speed to efficient implementations of full-precision, 8bit, and 4-bit quantized matrix multiplications. Our experiment shows our implementations of ternary and ternary-binary matrix multiplications to have almost the same inference time, and they are 3.6 times faster than full-precision, 2.5 times faster than 8bit quantized, and 1.4 times faster than 4-bit quantized matrix multiplication but 2.9 slower than binary matrix multiplication.

show abstract

Section: Introductionmentioning

confidence: 99%

Fast matrix multiplication for binary and ternary CNNs on ARM CPU

Trusov¹,

Limonova²,

Nikolaev³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This trend that is enabling the deployment of ML models in tiny, low-power, and low-latency IoT devices is called TinyML [ 7 , 32 ]. This alternative paradigm of TinyML tries to bring more intelligence to IoT devices, enabling the creation of novel applications that embed ML tasks in them.…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, by allowing some data analysis and interpretation to be performed locally and in real time at the collection point, services as these can translate into huge cost savings and better privacy protection [7]. Most of the work performed in the field of TinyML has been focused on the reduction and optimization of existing models, such as Artificial Neural Networks (ANNs), to fit into these tiny devices and commodity microcontrollers, despite their computational restrictions [7,32,33]. Additionally, for IoT scenarios, it can be argued that the algorithms should preferably work without prior knowledge of the data, i.e., unsupervised.…”

Section: Introductionmentioning

confidence: 99%

An Evolving TinyML Compression Algorithm for IoT Environments Based on Data Eccentricity

Signoretti

Silva

Andrade

et al. 2021

Sensors

View full text Add to dashboard Cite

Currently, the applications of the Internet of Things (IoT) generate a large amount of sensor data at a very high pace, making it a challenge to collect and store the data. This scenario brings about the need for effective data compression algorithms to make the data manageable among tiny and battery-powered devices and, more importantly, shareable across the network. Additionally, considering that, very often, wireless communications (e.g., low-power wide-area networks) are adopted to connect field devices, user payload compression can also provide benefits derived from better spectrum usage, which in turn can result in advantages for high-density application scenarios. As a result of this increase in the number of connected devices, a new concept has emerged, called TinyML. It enables the use of machine learning on tiny, computationally restrained devices. This allows intelligent devices to analyze and interpret data locally and in real time. Therefore, this work presents a new data compression solution (algorithm) for the IoT that leverages the TinyML perspective. The new approach is called the Tiny Anomaly Compressor (TAC) and is based on data eccentricity. TAC does not require previously established mathematical models or any assumptions about the underlying data distribution. In order to test the effectiveness of the proposed solution and validate it, a comparative analysis was performed on two real-world datasets with two other algorithms from the literature (namely Swing Door Trending (SDT) and the Discrete Cosine Transform (DCT)). It was found that the TAC algorithm showed promising results, achieving a maximum compression rate of 98.33%. Additionally, it also surpassed the two other models regarding the compression error and peak signal-to-noise ratio in all cases.

show abstract

“…Deste modo, a motivação desse trabalho vem da necessidade do desenvolvimento de algoritmos especializados que possam se adequar a esse cenário e suas limitações. Assim, sabe-se que o TinyML permite a implantação de modelos de aprendizado de máquina em dispositivos IoT e que apresentam baixo consumo de energia, poder de processamento e latência (Banbury et al, 2021;Gorospe et al, 2021). Além disso, técnicas de TinyML permitem que algumas análises e interpretações sejam realizadas localmente e em tempo real no ponto de coleta.…”

Section: Introdu ç ãOunclassified

Um Algoritmo de Compressão de Dados baseado em TinyML para Internet das Coisas

Signoretti

Silva

Andrade

et al. 2021

Procedings Do XV Simpósio Brasileiro De Automação Inteligente

View full text Add to dashboard Cite

With the advancement and mass adoption of Internet of Things (IoT) solutions, new challenges arise such as transmission and storage of the growing volume of data. Thus, it is clear that when devices transmit potentially irrelevant or redundant data, there is greater energy and processing expenditure, in addition to the unnecessary use of the communication channel. Thus, data compression solutions in edge computing devices themselves become increasingly attractive, enabling the elimination of samples that would have little or no contribution to the application, reducing the volume of data needed to represent the information. However, such devices present on the market today have serious limitations in terms of storage and processing power. In order to circumvent these limitations, the field of TinyML appears, which seeks ways to implement machine learning models in devices with low computational power. In this context, the present work proposes the development of a new online, unsupervised, and automatically adaptable data compression algorithm for IoT applications. To validate the proposal, a case study with data captured from vehicular sensors was used and preliminary results show that it is possible to reach 90% of compression rates, with mean absolute errors of 1,1 in the analyzed scenario.Resumo: Com o avanço e adoção em massa de soluções de Internet das Coisas (IoT), surgem novos desafios como transmissão e armazenamento do crescente volume de dados. Desta forma, percebe-se que quando dispositivos transmitem dados potencialmente irrelevantes ou redundantes, há um maior gasto de energia e processamento, além do uso desnecessário do canal de comunicação. Assim, soluções de compressão de dados nos próprios dispositivos (edge computing) se tornam cada vez mais atrativas, possibilitando a eliminação de amostras que teriam pouca ou nenhuma contribuição para a aplicação, reduzindo o volume de dados necessários para representar as informações. No entanto, tais dispositivos presentes hoje no mercado tem sérias limitações de armazenamento e poder de processamento. A fim de circundar tais limitações, surge o campo de TinyML, que busca maneiras de implementar modelos de aprendizado de máquina em dispositivos de baixo poder computacional. Nesse contexto, o presente trabalho propõe o desenvolvimento de um novo algoritmo de compressão de dados on-line, não supervisionado, e automaticamente adaptável para aplicações de IoT. Para validar a proposta um estudo de caso com dados capturados de sensores veiculares foi utilizado e resultados preliminares mostraram ser possível alcançar 90% de taxa de compressão, com um erro médio absoluto de 1.1 no cenário analisado.

show abstract

A Generalization Performance Study Using Deep Learning Networks in Embedded Systems

Cited by 23 publications

References 39 publications

Fast matrix multiplication for binary and ternary CNNs on ARM CPU

Fast matrix multiplication for binary and ternary CNNs on ARM CPU

An Evolving TinyML Compression Algorithm for IoT Environments Based on Data Eccentricity

Um Algoritmo de Compressão de Dados baseado em TinyML para Internet das Coisas

Contact Info

Product

Resources

About