A Machine Learning-Oriented Survey on Tiny Machine Learning (2023-09-21T00:00:00.000000Z)

TL;DR

This comprehensive survey wishes to provide an up-to-date overview focused on all the learning algorithms within TinyML-based solutions, and proposes a taxonomy that covers the learning panorama under the TinyML lens.

Abstract

The emergence of Tiny Machine Learning (TinyML) has positively revolutionized the field of Artificial Intelligence by promoting the joint design of resource-constrained IoT hardware devices and their learning-based software architectures. TinyML carries an essential role within the fourth and fifth industrial revolutions in helping societies, economies, and individuals employ effective AI-infused computing technologies (e.g., smart cities, automotive, and medical robotics). Given its multidisciplinary nature, the field of TinyML has been approached from many different angles: this comprehensive survey wishes to provide an up-to-date overview focused on all the learning algorithms within TinyML-based solutions. The survey is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodological flow, allowing for a systematic and complete literature survey. In particular, firstly, we will examine the three different workflows for implementing a TinyML-based system, i.e., ML-oriented, HW-oriented, and co-design. Secondly, we propose a taxonomy that covers the learning panorama under the TinyML lens, examining in detail the different families of model optimization and design, as well as the state-of-the-art learning techniques. Thirdly, this survey will present the distinct features of hardware devices and software tools that represent the current state-of-the-art for TinyML intelligent edge applications. Finally, we discuss the challenges and future directions.

Authors

Luigi Capogrosso

2 papers

Federico Cunico

3 papers

D. Cheng

1 papers

TL;DR

Abstract

Authors

References176 items

SensorGAN: A Novel Data Recovery Approach for Wearable Human Activity Recognition

Uncertainty-aware Energy Harvest Prediction and Management for IoT Devices

A hardware-aware neural architecture search algorithm targeting low-end microcontrollers

TyBox: An Automatic Design and Code Generation Toolbox for TinyML Incremental On-Device Learning

DDD TinyML: A TinyML-Based Driver Drowsiness Detection Model Using Deep Learning

TinyissimoYOLO: A Quantized, Low-Memory Footprint, TinyML Object Detection Network for Low Power Microcontrollers

A TinyML based Portable, Low-Cost Microwave Head Imaging System for Brain Stroke Detection

TinyReptile: TinyML with Federated Meta-Learning

Enhancing the Energy Efficiency and Robustness of tinyML Computer Vision Using Coarsely-quantized Log-gradient Input Images

Data Aware Neural Architecture Search

The Case for tinyML in Healthcare: CNNs for Real-Time On-Edge Blood Pressure Estimation

MetaLDC: Meta Learning of Low-Dimensional Computing Classifiers for Fast On-Device Adaption

An Adaptable and Unsupervised TinyML Anomaly Detection System for Extreme Industrial Environments

A Survey on RISC-V-Based Machine Learning Ecosystem

Self-supervised Learning: A Succinct Review

End to End Optimized Tiny Learning for Repositionable Walls in Maze Topologies

Self-Supervised Online and Lightweight Anomaly and Event Detection for IoT Devices

TinyML for Ultra-Low Power AI and Large Scale IoT Deployments: A Systematic Review

How TinyML Can be Leveraged to Solve Environmental Problems: A Survey

VeLO: Training Versatile Learned Optimizers by Scaling Up

Implementation and Optimization of Neural Networks for Tiny Hardware Devices

Tiny Machine Learning for Resource-Constrained Microcontrollers

Edge Impulse: An MLOps Platform for Tiny Machine Learning

ML-HW Co-Design of Noise-Robust TinyML Models and Always-On Analog Compute-in-Memory Edge Accelerator

MinUn: Accurate ML Inference on Microcontrollers

TinyRL: Towards Reinforcement Learning on Tiny Embedded Devices

Deep Reinforcement Learning-Based Energy-Efficient Edge Computing for Internet of Vehicles

Optimizing PhiNet architectures for the detection of urban sounds on low-end devices

Finding Meta Winning Ticket to Train Your MAML

Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approach

Incremental Online Learning Algorithms Comparison for Gesture and Visual Smart Sensors

Tiny-Sepformer: A Tiny Time-Domain Transformer Network for Speech Separation

Survey and Comparison of Milliwatts Micro controllers for Tiny Machine Learning at the Edge

Machine Learning for Microcontroller-Class Hardware: A Review

TinyML: Enabling of Inference Deep Learning Models on Ultra-Low-Power IoT Edge Devices for AI Applications

A TinyML Soft-Sensor Approach for Low-Cost Detection and Monitoring of Vehicular Emissions

Depth Pruning with Auxiliary Networks for Tinyml

Software Engineering Approaches for TinyML based IoT Embedded Vision: A Systematic Literature Review

Intelligence at the Extreme Edge: A Survey on Reformable TinyML

A General Survey on Attention Mechanisms in Deep Learning

AI on the edge: a comprehensive review

Overcoming Oscillations in Quantization-Aware Training

Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance

An Empirical Study of Low Precision Quantization for TinyML

A quantitative review of automated neural search and on-device learning for tiny devices

CFU Playground: Full-Stack Open-Source Framework for Tiny Machine Learning (TinyML) Acceleration on FPGAs

A review of neural architecture search

TinyML Meets IoT: A Comprehensive Survey

Mechanical Anomaly Detection on an Embedded Microcontroller

Tiny Generative Image Compression for Bandwidth-Constrained Sensor Applications

Enabling Hyperparameter Tuning of Machine Learning Classifiers in Production

TiWS-iForest: Isolation Forest in Weakly Supervised and Tiny ML scenarios

A Review of Machine Learning and TinyML in Healthcare

A review on TinyML: State-of-the-art and prospects

Resource Constrained CVD Classification Using Single Lead ECG On Wearable and Implantable Devices

Pruning Meta-Trained Networks for On-Device Adaptation

A TinyML Platform for On-Device Continual Learning With Quantized Latent Replays

MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer

Train++: An Incremental ML Model Training Algorithm to Create Self-Learning IoT Devices

IOT BASED SMART AGRICULTURE

Camera Distortion-aware 3D Human Pose Estimation in Video with Optimization-based Meta-Learning

Automated HW/SW Co-design for Edge AI: State, Challenges and Steps Ahead: Special Session Paper

Intermittent-Aware Neural Architecture Search

A Microcontroller is All You Need: Enabling Transformer Execution on Low-Power IoT Endnodes

Machine Learning and Deep Learning: Open Issues and Future Research Directions for the Next 10 Years

AnoML-IoT: An end to end re-configurable multi-protocol anomaly detection pipeline for Internet of Things

MLPerf Tiny Benchmark

An Unsupervised TinyML Approach Applied for Pavement Anomalies Detection Under the Internet of Intelligent Vehicles

An Efficient and Low-Power MLP Accelerator Architecture Supporting Structured Pruning, Sparse Activations and Asymmetric Quantization for Edge Computing

Accelerating DNNs inference with predictive layer fusion

Measuring what Really Matters: Optimizing Neural Networks for TinyML

OutlierNets: Highly Compact Deep Autoencoder Network Architectures for On-Device Acoustic Anomaly Detection

A Survey of Quantization Methods for Efficient Neural Network Inference

Applications of the Internet of Things (IoT) in Smart Logistics: A Comprehensive Survey

TinyOL: TinyML with Online-Learning on Microcontrollers

General Instance Distillation for Object Detection