TinyFedTL: Federated Transfer Learning on Ubiquitous Tiny IoT Devices

Kopparapu, Kavya; Lin, Eric W.; Breslin, John G.; Sudharsan, Bharath

doi:10.1109/percomworkshops53856.2022.9767250

Cited by 34 publications

(21 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1) robust to laggards or client disconnections [190]; 2) achieves similar accuracy across all devices [190]; 3) includes device-specific model pruning to improve communication and training cost [192], [194]; 4) uses transfer learning or fine-tuning for local model updates to save memory and build personalized models [195], [197]; 5) uses knowledge distillation to aggregate class probabilities instead of weights [197].…”

Section: B Federated Learningmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning for Microcontroller-Class Hardware: A Review

2022

View full text Add to dashboard Cite

The advancements in machine learning (ML) opened a new opportunity to bring intelligence to the low-end Internet-of-Things (IoT) nodes, such as microcontrollers. Conventional ML deployment has high memory and computes footprint hindering their direct deployment on ultraresource-constrained microcontrollers. This article highlights the unique requirements of enabling onboard ML for microcontroller-class devices. Researchers use a specialized model development workflow for resource-limited applications to ensure that the compute and latency budget is within the device limits while still maintaining the desired performance. We characterize a closed-loop widely applicable workflow of ML model development for microcontroller-class devices and show that several classes of applications adopt a specific instance of it. We present both qualitative and numerical insights into different stages of model development by showcasing several use cases. Finally, we identify the open research challenges and unsolved questions demanding careful considerations moving forward.

show abstract

Section: B Federated Learningmentioning

confidence: 99%

“…5) Client Hardware and Supported Languages: Finally, the FL frameworks must support a wide variety of clients with different processors and operating languages. Among the different frameworks, Flower [190], PruneFL [194], and TinyFedTL [195] were tested on microcontrollers, supporting Python, Java, and C++.…”

Section: B Federated Learningmentioning

confidence: 99%

Machine Learning for Microcontroller-Class Hardware: A Review

2022

View full text Add to dashboard Cite

show abstract

“…While TinyML can allow local execution of the models, FL can provide means for exploiting cross-device knowledge to improve the prediction model itself. To the best of our knowledge, the only attempt made to combine these two is in [26] and this is still an open area for future research.…”

Section: B Tinyml For Federated Learningmentioning

confidence: 99%

Unlocking Edge Intelligence Through Tiny Machine Learning (TinyML)

et al. 2022

View full text Add to dashboard Cite

Machine Learning (ML) on the edge is key for enabling a new breed of IoT and autonomous system applications. The departure from the traditional cloud-centric architecture means that new deployments can be more power-efficient, provide better privacy and reduced latency for inference. At the core of this paradigm is TinyML, a framework allowing the execution of ML models on low-power embedded devices. TinyML allows importing pre-trained ML models on the edge for providing ML-as-a-Service (MLaaS) to IoT devices. This article presents a comprehensive overview of Tiny MLaaS (TMLaaS) architecture. The TMLaaS architecture inherently presents several design trade-offs in terms of energy consumption, security, privacy, and latency. We also present how TMLaaS architecture can be implemented, deployed, and maintained for large scale IoT deployment. The feasibility of implementation for the TMLaaS architecture has been demonstrated with the help of a case study.

show abstract

“…This procedure reduces the computational resources required to train a new model. In [89], the authors presented a method named federated transfer learning on tiny devices (TinyFedTL), whereby they implemented their own fully connected layer inference and backpropagation update between an Arduino Nano 33 BLE Sense microcontrollers and a local server. As a result, they managed to train an ML model without sending raw data to the server; only the weights and bias data had to be sent between the client nodes and the server.…”

Section: Emerging Techniques Of Tinymlmentioning

confidence: 99%

Tiny Machine Learning for Resource-Constrained Microcontrollers

Immonen

Hämäläinen

2022

Journal of Sensors

View full text Add to dashboard Cite

We use 250 billion microcontrollers daily in electronic devices that are capable of running machine learning models inside them. Unfortunately, most of these microcontrollers are highly constrained in terms of computational resources, such as memory usage or clock speed. These are exactly the same resources that play a key role in teaching and running a machine learning model with a basic computer. However, in a microcontroller environment, constrained resources make a critical difference. Therefore, a new paradigm known as tiny machine learning had to be created to meet the constrained requirements of the embedded devices. In this review, we discuss the resource optimization challenges of tiny machine learning and different methods, such as quantization, pruning, and clustering, that can be used to overcome these resource difficulties. Furthermore, we summarize the present state of tiny machine learning frameworks, libraries, development environments, and tools. The benchmarking of tiny machine learning devices is another thing to be concerned about; these same constraints of the microcontrollers and diversity of hardware and software turn to benchmark challenges that must be resolved before it is possible to measure performance differences reliably between embedded devices. We also discuss emerging techniques and approaches to boost and expand the tiny machine learning process and improve data privacy and security. In the end, we form a conclusion about tiny machine learning and its future development.

show abstract

TinyFedTL: Federated Transfer Learning on Ubiquitous Tiny IoT Devices

Cited by 34 publications

References 3 publications

Machine Learning for Microcontroller-Class Hardware: A Review

Machine Learning for Microcontroller-Class Hardware: A Review

Unlocking Edge Intelligence Through Tiny Machine Learning (TinyML)

Tiny Machine Learning for Resource-Constrained Microcontrollers

Contact Info

Product

Resources

About