research-article

EchoSensor: Fine-grained Ultrasonic Sensing for Smart Home Intrusion Detection

Authors:

Xu YuanAuthors Info & Claims

ACM Transactions on Sensor Networks, Volume 20, Issue 1

Article No.: 10, Pages 1 - 24

https://doi.org/10.1145/3615658

Published: 19 October 2023 Publication History

Abstract

This article presents the design and implementation of a novel intrusion detection system, called EchoSensor, which leverages speakers and microphones in smart home devices to capture human gait patterns for individual identification. EchoSensor harnesses the speaker to send inaudible acoustic signals (around 20�kHz) and utilizes the microphone to capture the reflected signals. As the reflected signals have unique variations in the Doppler shift respective to the gaits of different people, EchoSensor is able to profile human gait patterns from the generated spectrograms. To mine the gait information, we first propose a two-stage interference cancellation scheme to remove the background noise and environmental interference, followed by a new method to detect the starting point of walking and estimate the gait cycle time. We then perform the fine-grained analysis of the spectrograms to extract a series of features. In the end, machine learning is employed to construct an identifier for individual recognition. We implement the EchoSensor system and deploy it under different household environments to conduct intrusion detection tasks. Extensive experimental results have demonstrated that EchoSensor can achieve the averaged Intruder Gait Detection Rate (IDR) and True Family Member Gait Detection Rate (TFR) of 92.7% and 91.9%, respectively.

References

[1]

M. Umair Bin Altaf, Taras Butko, and Biing-Hwang Fred Juang. 2015. Acoustic gaits: Gait analysis with footstep sounds. IEEE Transactions on Biomedical Engineering 62, 8 (2015), 2001–2011.

[2]

Imed Bouchrika, Michaela Goffredo, John Carter, and Mark Nixon. 2011. On using gait in forensic biometrics. Journal of Forensic Sciences 56, 4 (2011), 882–889.

[3]

Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 3 (2011), 27.

Digital Library

[4]

Jagmohan Chauhan, Yining Hu, Suranga Seneviratne, Archan Misra, Aruna Seneviratne, and Youngki Lee. 2017. BreathPrint: Breathing acoustics-based user authentication. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 278–291.

Digital Library

[5]

Gilles Degottex. 2010. Glottal Source and Vocal-tract Separation. Ph.D. Dissertation.

[6]

Davrondzhon Gafurov. 2007. A survey of biometric gait recognition: Approaches, security, and challenges. In Proceedings of the Annual Norwegian Computer Science Conference. Annual Norwegian Computer Science Conference Norway, 19–21.

[7]

Davrondzhon Gafurov, Kirsi Helkala, and Torkjel Søndrol. 2006. Biometric gait authentication using accelerometer sensor. JCP 1, 7 (2006), 51–59.

[8]

Daniel Graham, George Simmons, David T. Nguyen, and Gang Zhou. 2015. A software-based sonar ranging sensor for smart phones. IEEE Internet of Things Journal 2, 6 (2015), 479–489.

[9]

Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. Soundwave: Using the Doppler effect to sense gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1911–1914.

Digital Library

[10]

Ju Han and Bir Bhanu. 2006. Individual recognition using gait energy image. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 2 (2006), 316–322.

Digital Library

[11]

Spencer Ives. 2018. Parks Associates Predicts about 27 Percent of U.S. Households to have Security by 2021. Retrieved March 15, 2019 from http://www.securitysystemsnews.com/article/parks-associates-predicts-about-27-percent-us-households-have-security-2021

[12]

Jam Jenkins and Carla Ellis. 2007. Using ground reaction forces from gait analysis: Body mass as a weak biometric. In Proceedings of the International Conference on Pervasive Computing. Springer, 251–267.

[13]

Kaustubh Kalgaonkar and Bhiksha Raj. 2007. Acoustic Doppler sonar for gait recogination. In Proceedings of the 2007 IEEE Conference on Advanced Video and Signal Based Surveillance. IEEE, 27–32.

Digital Library

[14]

A. J. Lawrance and P. A. W. Lewis. 1977. An exponential moving-average sequence and point process (EMA1). Journal of Applied Probability 14, 1 (1977), 98–113.

[15]

Hoang Thanh Le, Son Lam Phung, and Abdesselam Bouzerdoum. 2018. Human gait recognition with micro-doppler radar and deep autoencoder. In Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR’18). IEEE, 3347–3352.

[16]

Dong Li, Shirui Cao, Sunghoon Ivan Lee, and Jie Xiong. 2022. Experience: Practical problems for acoustic sensing. In Proceedings of the 28th Annual International Conference on Mobile Computing And Networking. 381–390.

Digital Library

[17]

Hong Li, Yunhua He, Limin Sun, Xiuzhen Cheng, and Jiguo Yu. 2016. Side-channel information leakage of encrypted video stream in video surveillance systems. In Proceedings of the IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications. IEEE, 1–9.

Digital Library

[18]

Jinyang Li, Zhenyu Li, Gareth Tyson, and Gaogang Xie. 2020. Your privilege gives your privacy away: An analysis of a home security camera service. In Proceedings of the IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 387–396.

Digital Library

[19]

Li Lu, Jiadi Yu, Yingying Chen, Hongbo Liu, Yanmin Zhu, Yunfei Liu, and Minglu Li. 2018. Lippass: Lip reading-based user authentication on smartphones leveraging acoustic signals. In Proceedings of the IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, 1466–1474.

Digital Library

[20]

Yongsen Ma, Gang Zhou, and Shuangquan Wang. 2019. WiFi sensing with channel state information: A survey. ACM Computing Surveys 52, 3 (2019), 46.

[21]

Roger G. T. Mello, Liliam F. Oliveira, and Jurandir Nadal. 2007. Digital Butterworth filter for subtracting noise from low magnitude surface electromyogram. Computer Methods and Programs in Biomedicine 87, 1 (2007), 28–35.

Digital Library

[22]

Soumik Mondal, Anup Nandy, Pavan Chakraborty, and G. C. Nandi. 2012. Gait-based personal identification system using rotation sensor. Journal of Emerging Trends in Computing and Information Sciences 3, 2 (2012), 395–402.

[23]

Alvaro Muro-De-La-Herran, Begonya Garcia-Zapirain, and Amaia Mendez-Zorrilla. 2014. Gait analysis methods: An overview of wearable and non-wearable systems, highlighting clinical applications. Sensors 14, 2 (2014), 3362–3394.

[24]

Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. Fingerio: Using active sonar for fine-grained finger tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1515–1525.

Digital Library

[25]

Rajalakshmi Nandakumar, Alex Takakuwa, Tadayoshi Kohno, and Shyamnath Gollakota. 2017. Covertband: Activity information leakage using music. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 87.

Digital Library

[26]

Thanh Trung Ngo, Yasushi Makihara, Hajime Nagahara, Yasuhiro Mukaigawa, and Yasushi Yagi. 2014. The largest inertial sensor-based gait database and performance evaluation of gait-based personal authentication. Pattern Recognition 47, 1 (2014), 228–237.

Digital Library

[27]

G. Nicholas. 2010. College Physics: Reasoning and Relationships. Brooks/Cole.

[28]

Mark S. Nixon and John N. Carter. 2006. Automatic recognition by gait. Proceedings of the IEEE 94, 11 (2006), 2013–2024.

[29]

Alan Oppenheim and Ronald Schafer. 1968. Homomorphic analysis of speech. IEEE Transactions on Audio and Electroacoustics 16, 2 (1968), 221–226.

[30]

Robert J. Orr and Gregory D. Abowd. 2000. The smart floor: A mechanism for natural user identification and tracking. In Proceedings of the CHI’00 Extended Abstracts on Human Factors in Computing Systems. ACM, 275–276.

Digital Library

[31]

Shijia Pan, Ningning Wang, Yuqiu Qian, Irem Velibeyoglu, Hae Young Noh, and Pei Zhang. 2015. Indoor person identification through footstep induced structural vibration. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. ACM, 81–86.

Digital Library

[32]

Yanzhi Ren, Yingying Chen, Mooi Choo Chuah, and Jie Yang. 2013. Smartphone-based user verification leveraging gait recognition for mobile healthcare systems. In Proceedings of the 2013 IEEE International Conference on Sensing, Communications and Networking (SECON). IEEE, 149–157.

[33]

Chandra Rishi and Huffman Scott. 2018. How Google Home and the Google Assistant Helped you Get More Done in 2017. Retrieved March 15, 2019 from https://www.blog.google/products/assistant/how-google-home-and-google-assistant-helped-you-get-more-done-in-2017

[34]

Claude Elwood Shannon. 1998. Communication in the presence of noise. Proceedings of the IEEE 86, 2 (1998), 447–457.

[35]

Ke Sun, Ting Zhao, Wei Wang, and Lei Xie. 2018. Vskin: Sensing touch gestures on surfaces of mobile devices using acoustic signals. In Proceedings of the ACM Annual International Conference on Mobile Computing and Networking (MobiCom’18). 591–605.

Digital Library

[36]

Thiago Teixeira, Deokwoo Jung, Gershon Dublon, and Andreas Savvides. 2009. PEM-ID: Identifying people by gait-matching using cameras and wearable accelerometers. In Proceedings of the 2009 3rd ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC). IEEE, 1–8.

[37]

Inc TranSafety. 1997. Study compares older and younger pedestrian walking speeds. Road Management and Engineering Journal (1997). https://web.archive.org/web/20090703084118 http://www.usroads.com/journals/p/rej/9710/re971001.htm. Access date March 15, 2019.

[38]

Junia Valente, Keerthi Koneru, and Alvaro Cardenas. 2019. Privacy and security in Internet-connected cameras. In Proceedings of the 2019 IEEE International Congress on Internet of Things (ICIOT’19). IEEE, 173–180.

[39]

Ph Van Dorp and F. C. A. Groen. 2008. Feature-based human motion parameter estimation with radar. IET Radar, Sonar and Navigation 2, 2 (2008), 135–145.

[40]

Ashley D. Waite. 2002. Sonar for Practising Engineers. John Wiley and Sons.

[41]

Wei Wang, Alex X. Liu, and Muhammad Shahzad. 2016. Gait recognition using wifi signals. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 363–373.

Digital Library

[42]

Yingxue Wang, Yanan Chen, Md Zakirul Alam Bhuiyan, Yu Han, Shenghui Zhao, and Jianxin Li. 2018. Gait-based human identification using acoustic sensor and deep neural network. Future Generation Computer Systems 86 (2018), 1228–1237.

Digital Library

[43]

Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemometrics and Intelligent Laboratory Systems 2, 1-3 (1987), 37–52.

[44]

Dan Wu, Daqing Zhang, Chenren Xu, Hao Wang, and Xiang Li. 2017. Device-free WiFi human sensing: From pattern-based to model-based approaches. IEEE Communications Magazine 55, 10 (2017), 91–97.

[45]

Wei Xu, ZhiWen Yu, Zhu Wang, Bin Guo, and Qi Han. 2019. Acousticid: Gait-based human identification using acoustic signal. Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies 3, 3 (2019), 1–25.

Digital Library

[46]

Yunze Zeng, Parth H. Pathak, and Prasant Mohapatra. 2016. WiWho: Wifi-based person identification in smart spaces. In Proceedings of the 15th International Conference on Information Processing in Sensor Networks. IEEE.

[47]

Jin Zhang, Bo Wei, Wen Hu, and Salil S. Kanhere. 2016. Wifi-id: Human identification using wifi signal. In Proceedings of the 2016 International Conference on Distributed Computing in Sensor Systems (DCOSS). IEEE, 75–82.

[48]

Linghan Zhang, Sheng Tan, and Jie Yang. 2017. Hearing your voice is not enough: An articulatory gesture-based liveness detection for voice authentication. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 57–71.

Digital Library

[49]

Yuting Zhang, Gang Pan, Kui Jia, Minlong Lu, Yueming Wang, and Zhaohui Wu. 2015. Accelerometer-based gait recognition by sparse representation of signature points with clusters. IEEE Transactions on Cybernetics 45, 9 (2015), 1864–1875.

[50]

Zhaonian Zhang, Philippe O. Pouliquen, Allen Waxman, and Andreas G. Andreou. 2007. Acoustic micro-Doppler radar for human gait imaging. The Journal of the Acoustical Society of America 121, 3 (2007), EL110–EL113.

[51]

Peijun Zhao, Chris Xiaoxuan Lu, Jianan Wang, Changhao Chen, Wei Wang, Niki Trigoni, and Andrew Markham. 2019. mid: Tracking and identifying people with millimeter wave radar. In Proceedings of the 2019 15th International Conference on Distributed Computing in Sensor Systems (DCOSS’19). IEEE, 33–40.

Cited By

Wang DZhang XWang KWang LFan XZhang Y(2024)RDGait: A mmWave Based Gait User Recognition System for Complex Indoor Environments Using Single-chip RadarProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785528:3(1-31)Online publication date: 9-Sep-2024
https://doi.org/10.1145/3678552
Wang XZhang DZhan FXie XHuang PHu YChen Y(2024)RoFi: Robust WiFi Intrusion Detection via Distribution MatchingICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446917(166-170)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10446917
Xu XZhang XBao ZYu XYin YYang XNiu Q(2023)Training-Free Acoustic-Based Hand Gesture Tracking on Smart SpeakersApplied Sciences10.3390/app13211195413:21(11954)Online publication date: 1-Nov-2023
https://doi.org/10.3390/app132111954

Index Terms

EchoSensor: Fine-grained Ultrasonic Sensing for Smart Home Intrusion Detection
1. Human-centered computing
  1. Ubiquitous and mobile computing

Recommendations

Rule generalisation in intrusion detection systems using SNORT

Intrusion Detection Systems (IDSs) provide an important layer of security for computer systems and networks. An IDS's responsibility is to detect suspicious or unacceptable system and network activity and to alert a systems administrator to this ...
Network Intrusion Detection: Automated and Manual Methods Prone to Attack and Evasion

In this article, the authors describe common intrusion detection techniques, NIDS evasion methods, and how NIDSs detect intrusions. Additionally, we introduce new evasion methods, present test results for confirming attack outcomes based on server ...
Syntax vs. semantics: competing approaches to dynamic network intrusion detection

Malicious network traffic, including widespread worm activity, is a growing threat to internet-connected networks and hosts. In this paper, we consider both syntax and semantics based approaches for dynamic network intrusion detection. The semantics-...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Sensor Networks

ACM Transactions on Sensor Networks Volume 20, Issue 1

January 2024

717 pages

EISSN:1550-4867

DOI:10.1145/3618078

Editor:
Yunhao Liu
Tsinghua University, China

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 19 October 2023

Online AM: 12 August 2023

Accepted: 25 July 2023

Revised: 14 January 2023

Received: 17 February 2022

Published in�TOSN�Volume 20, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF
BoRSF

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
343
Total Downloads

Downloads (Last 12 months)280
Downloads (Last 6 weeks)32

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang DZhang XWang KWang LFan XZhang Y(2024)RDGait: A mmWave Based Gait User Recognition System for Complex Indoor Environments Using Single-chip RadarProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785528:3(1-31)Online publication date: 9-Sep-2024
https://doi.org/10.1145/3678552
Wang XZhang DZhan FXie XHuang PHu YChen Y(2024)RoFi: Robust WiFi Intrusion Detection via Distribution MatchingICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446917(166-170)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10446917
Xu XZhang XBao ZYu XYin YYang XNiu Q(2023)Training-Free Acoustic-Based Hand Gesture Tracking on Smart SpeakersApplied Sciences10.3390/app13211195413:21(11954)Online publication date: 1-Nov-2023
https://doi.org/10.3390/app132111954

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents