Introduction to the Special Section on Contact-free Smart Sensing in AIoT

Artificial Intelligence (AI) and the Internet of Things (IoT) are two powerful forces that have been reshaping our world in recent years. When they converge, they create a new field of AIoT that enables ubiquitous intelligence through the integration of smart algorithms and connected devices. One of the key enablers of AIoT is contact-free sensing, which leverages the availability of portable and highly integrated WiFi, radar, and sonar-style devices to monitor humans and environments without physical contact. This technology has transformed the traditional computer vision-based paradigms and opened up novel possibilities for data collection and analysis. However, contact-free sensing also poses new challenges and risks for AIoT applications. The dynamic and complex wireless environments require innovative solutions for efficient data processing and interpretation. The security and privacy issues of WiFi, radar, and sonar-enabled sensing devices also demand urgent attention, as they may expose sensitive information to malicious attacks. Therefore, it is imperative to explore the potential and pitfalls of contact-free sensing in AIoT and to develop effective strategies for ensuring the robustness and reliability of AIoT applications.

This special issue is dedicated to highlighting the cutting-edge methods and latest research in the field of contact-free sensing, which leverages WiFi, radar, and sonar-style devices to monitor humans and environments without physical contact. The main focus of this issue is to explore the latest machine learning analytics to extract information from the sensory data and to investigate the potential risks and countermeasures to ensure the security and privacy of sensing devices. The call for papers attracted with 44 submissions and after a rigorous review, 18 papers have been accepted for this special issue. A brief summary of some papers in this special issue is presented in the following:

In “Feasibility of Remote Blood Pressure Estimation via Narrow-band Multi-wavelength Pulse Transit Time,” the authors investigate the feasibility of estimating blood pressure (BP) via pulse transit time (PTT) in a novel remote single-site manner using a modified RGB camera. A narrow-band triple band-pass filter makes it possible to measure the PTT between different skin layers, harvesting information from green and near-infrared wavelengths. They design a color-channel model and a novel channel-separation method to further resolve the inter-channel influence and band overlap. The results showed a good absolute Pearson’s correlation coefficient between both MW PTT and systolic BP as well as diastolic BP, pointing to the feasibility of the proposed novel remote MW BP estimation via PTT.

In “LiteWiSys: A Lightweight System for WiFi-based Dual-task Action Perception,” Sheng et al. propose a lightweight system named LiteWiSys that can simultaneously detect and recognize WiFi-based human actions. This work addresses two major drawbacks of existing methods: heavy empirical dependency and large computational complexity. Specifically, the authors reduce noise and information redundancy by compressing sub-carriers. Then, LiteWiSys integrates deep separable convolution and channel shuffle mechanisms into a multi-scale convolutional backbone structure. By feature channel split, two network branches are obtained and further trained with a joint loss function for dual tasks. Extensive experimental studies on different datasets in various scenes demonstrate that LiteWiSys achieves promising precision with a lower complexity compared to existing WiFi sensing systems.

To solve the class imbalance problem, the authors of “TFSemantic: A Time-Frequency Semantic GAN Framework for Imbalanced Classification Using Radio Signals” introduce a time-frequency semantic generative adversarial network (GAN) framework (i.e., TFSemantic) to address the imbalanced classification problem in human activity recognition using radio frequency (RF) signals. Specifically, the TFSemantic framework can learn semantic features from the minority classes and then generate high-quality signals to restore data balance. The authors evaluate the effectiveness of the proposed TFSemantic framework using different RF datasets (i.e., WiFi, RFID, UWB, and mmWave), which show that it outperforms several state-of-the-art methods.

In “TomFi: Small Object Tracking Using Commodity WiFi,” Zhang et al. propose a WiFi-based active small object tracking system, namely TomFi. It first employs a detection model to detect the appearance of the rat and then localizes it based on a two-branch localization model, which can solve the information loss and distortion problem. Experimental results show that TomFi can achieve an excellent detection success rate and a centimeter-level localization accuracy in real time.

The authors of “Afitness: Fitness Monitoring on Smart Devices via Acoustic Motion Images” propose an acoustic-based sensing system that enables non-intrusive, passive, and high-precision fitness detection. Afitness can utilize pulse compression to generate high-precision motion distance images that can be visually recognized. Then the authors exploit incremental learning techniques that allow Afitness to improve the portability of the system and recognize new actions. Overall, Afitness achieves acoustic signal interpretability and environmental reliability detection.

In “UltraCLR: Contrastive Representation Learning Framework for Ultrasound-based Sensing,” Wang et al. propose a new contrastive learning framework that fuses dual modulation ultrasonic sensing signals to enhance gesture representation. UltraCLR aims to autonomously learn a robust gesture signal representation that can benefit all tasks from low-cost unlabeled continuous signals. It utilizes the STFT heatmap as a secondary input and leverages the contrastive learning framework to improve the quality of the Channel Impulsive Response heatmap input representations. UltraCLR achieves more than 85% reduction in computational complexity and over 9\(\times\) improvement in inference speed.

In AIoT, the areas of privacy protection and secure storage are gaining attention. In “Efficient Task-Driven Video Data Privacy Protection for Smart Camera Surveillance System,” Wang et al. design a task-driven and efficient video privacy protection mechanism for a better tradeoff between privacy and data usability. Class Activation Mapping is utilized to protect privacy while preserving data usability. To improve the efficiency, the authors exploit the motion vector and residual matrix produced during video codec. This work outperforms the ROI-based methods in data protection while preserving data usability. On the other hand, to ensure the security of data from contact-free smart sensing devices, the authors of “Privacy-Enhanced Cooperative Storage Scheme for Contact-free Sensory Data in AIoT with Efficient Synchronization” propose a Cloud-Edge-End cooperative storage scheme. The processed sensory data are stored separately in the three layers by utilizing a well-designed data partitioning strategy. Then, in combination with the Cloud-Edge-End cooperation model, this work proposes a delta-based data update method and extends it into a hybrid update mode to improve the synchronization efficiency. The experimental results show that the proposed cooperative storage method can resist various security threats in adverse situations.

In “Water Salinity Sensing with UAV-Mounted IR-UWB Radar,” the authors propose a novel water salinity sensing system called USalt, which leverages the high mobility of UAV and the contactless sensing ability of IR-UWB radar and realizes fast and accurate water salinity sensing for surface water. Specifically, they design novel methods to eliminate the contamination in raw received radar signals and extract salinity-related features from radar signals. The authors also adopt a neural network model named ssNet to precisely estimate water salinity using the extracted features. Moreover, they customize meta-learning and design a meta-learning framework called mssNet to efficiently adapt ssNet to different environments. Extensive real-world experiments carried out by their UAV-based system illustrate that USalt can accurately sense the salinity of water.

The authors of “Wavoice: A mmWave-assisted Noise-resistant Speech Recognition System” propose Wavoice, the first noise-resistant multi-modal speech recognition system that fuses millimeter-wave signals and audio signals from a microphone. Wavoice facilitates real-time noise-resistant voice activity detection and user targeting from multiple speakers. Additionally, they introduce two novel modules for multi-modal fusion that are embedded into the neural network, leading to accurate speech recognition. Extensive experiments demonstrate the effectiveness of Wavoice under adverse conditions.

In “LightGyro: A Batteryless Orientation Measuring Scheme Based on Light Reflection,” Guo et al. present LightGyro, a low-cost and efficient batteryless scheme to measure the orientation, in which they attach a reflective film to the target device and use a camera to capture the light spot on the reflective film. To solve the lack of distinctive features of a single light spot, the authors switch light sources on and off to regulate the appearance of light spots and utilize frame subtraction to extract light spots. A light array-based reflection model is proposed to extract the depth of field from the relative positions of multiple light spots to address the issue of depth of field of light spot loss in the process of camera projection. The experimental results show the effectiveness of utilizing reflection to measure orientation.

We are grateful to the authors for their insightful contributions, the reviewers for their rigorous work, and the editorial team of the journal for their excellent support and cooperation. We hope that this Special Issue will serve as a catalyst, sparking new avenues of research and development in the realm of Contact-free Smart Sensing in the IoT.