From Wrist to World: Harnessing Wearable IMU Sensors and TinyML to Enable Smart Environment Interactions

In this demonstration paper, we introduce a pioneering method to elevate smart environment interactions using IMU sensor data from smartwatches or fitness trackers. Central to our approach is the classification of hand gestures, facilitating a seamless synergy between human gestures and device interactions. Through the integration of Tiny Machine Learning (TinyML) techniques, our system expands wearables’ functionalities beyond traditional fitness monitoring to include gesture-based controls. The process comprises data collection, model training, and subsequent deployment to the wearable’s microcontroller (MCU) for on-device machine learning inference. Such on-device computing ensures data privacy, as no data is transmitted externally, and guarantees low latency for real-time responses. Preliminary evaluations showcase the system’s effectiveness in tasks like controlling PowerPoint presentations, managing table lamp operations, and interpreting specific hand gestures for device commands. Notably, our system achieves commendable gesture recognition accuracy while optimizing resource usage, underscoring its vast potential in diverse smart environment settings.