research-article

Open access

CrossHAR: Generalizing Cross-dataset Human Activity Recognition via Hierarchical Self-Supervised Pretraining

Authors:

Desheng ZhangAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 8, Issue 2

Article No.: 64, Pages 1 - 26

https://doi.org/10.1145/3659597

Published: 15 May 2024 Publication History

Abstract

The increasing availability of low-cost wearable devices and smartphones has significantly advanced the field of sensor-based human activity recognition (HAR), attracting considerable research interest. One of the major challenges in HAR is the domain shift problem in cross-dataset activity recognition, which occurs due to variations in users, device types, and sensor placements between the source dataset and the target dataset. Although domain adaptation methods have shown promise, they typically require access to the target dataset during the training process, which might not be practical in some scenarios. To address these issues, we introduce CrossHAR, a new HAR model designed to improve model performance on unseen target datasets. CrossHAR involves three main steps: (i) CrossHAR explores the sensor data generation principle to diversify the data distribution and augment the raw sensor data. (ii) CrossHAR then employs a hierarchical self-supervised pretraining approach with the augmented data to develop a generalizable representation. (iii) Finally, CrossHAR fine-tunes the pretrained model with a small set of labeled data in the source dataset, enhancing its performance in cross-dataset HAR. Our extensive experiments across multiple real-world HAR datasets demonstrate that CrossHAR outperforms current state-of-the-art methods by 10.83% in accuracy, demonstrating its effectiveness in generalizing to unseen target datasets.

References

[1]

2023. Android Document. 2023. https://developer.android.com/develop/sensors-and-location/sensors/sensors_overview.

[2]

2023. Android Studio. 2023. https://developer.android.com/studio.

[3]

Alireza Abedin, Mahsa Ehsanpour, Qinfeng Shi, Hamid Rezatofighi, and Damith C Ranasinghe. 2021. Attend and discriminate: Beyond the state-of-the-art for human activity recognition using wearable sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (2021), 1--22.

Digital Library

[4]

Sejal Bhalla, Mayank Goel, and Rushil Khurana. 2022. IMU2Doppler: Cross-Modal Domain Adaptation for Doppler-Based Activity Recognition Using IMU Data. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 4, Article 145 (dec 2022), 20 pages. https://doi.org/10.1145/3494994

Digital Library

[5]

Eoin Brophy, Zhengwei Wang, Qi She, and Tom�s Ward. 2023. Generative Adversarial Networks in Time Series: A Systematic Literature Review. ACM Comput. Surv. 55, 10, Article 199 (feb 2023), 31 pages. https://doi.org/10.1145/3559540

Digital Library

[6]

Sara Caramaschi, Gabriele B Papini, and Enrico G Caiani. 2023. Device Orientation Independent Human Activity Recognition Model for Patient Monitoring Based on Triaxial Acceleration. Applied Sciences 13, 7 (2023), 4175.

[7]

Youngjae Chang, Akhil Mathur, Anton Isopoussu, Junehwa Song, and Fahim Kawsar. 2020. A Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity Recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 1, Article 39 (mar 2020), 30 pages. https://doi.org/10.1145/3380985

Digital Library

[8]

Kaixuan Chen, Dalin Zhang, Lina Yao, Bin Guo, Zhiwen Yu, and Yunhao Liu. 2021. Deep Learning for Sensor-Based Human Activity Recognition: Overview, Challenges, and Opportunities. ACM Comput. Surv. 54, 4, Article 77 (may 2021), 40 pages. https://doi.org/10.1145/3447744

Digital Library

[9]

Ling Chen, Yi Zhang, and Liangying Peng. 2020. METIER: A Deep Multi-Task Learning Based Activity and User Recognition Model Using Wearable Sensors. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 1, Article 5 (mar 2020), 18 pages. https://doi.org/10.1145/3381012

Digital Library

[10]

Si-An Chen, Chun-Liang Li, Nate Yoder, Sercan O Arik, and Tomas Pfister. 2023. Tsmixer: An all-mlp architecture for time series forecasting. arXiv preprint arXiv:2303.06053 (2023).

[11]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597--1607.

[12]

Ranak Roy Chowdhury, Jiacheng Li, Xiyuan Zhang, Dezhi Hong, Rajesh K Gupta, and Jingbo Shang. 2023. PrimeNet: Pre-Training for Irregular Multivariate Time Series. In Proceedings of the AAAI Conference on Artificial Intelligence.

Digital Library

[13]

Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. 2023. SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling. arXiv preprint arXiv:2302.00861 (2023).

[14]

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).

[15]

Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, Chee Keong Kwoh, Xiaoli Li, and Cuntai Guan. 2021. Time-series representation learning via temporal and contextual contrasting. arXiv preprint arXiv:2106.14112 (2021).

[16]

Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, Chee-Keong Kwoh, Xiaoli Li, and Cuntai Guan. 2023. Self-supervised contrastive representation learning for semi-supervised time-series classification. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).

Digital Library

[17]

John Cristian Borges Gamboa. 2017. Deep learning for time-series analysis. arXiv preprint arXiv:1701.01887 (2017).

[18]

Ziqi Gao, Yuntao Wang, Jianguo Chen, Junliang Xing, Shwetak Patel, Xin Liu, and Yuanchun Shi. 2023. MMTSA: Multi-Modal Temporal Segment Attention Network for Efficient Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 3 (2023), 1--26.

Digital Library

[19]

Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. 2023. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 15180--15190.

[20]

Baoshen Guo, Weijian Zuo, Shuai Wang, Wenjun Lyu, Zhiqing Hong, Yi Ding, Tian He, and Desheng Zhang. 2022. WePos: Weak-Supervised Indoor Positioning with Unlabeled WiFi for On-Demand Delivery. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 2, Article 54 (jul 2022), 25 pages. https://doi.org/10.1145/3534574

Digital Library

[21]

Juan Haladjian. 2019. The wearables development toolkit: an integrated development environment for activity recognition applications. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 4 (2019), 1--26.

Digital Library

[22]

Harish Haresamudram, Irfan Essa, and Thomas Pl�tz. 2021. Contrastive Predictive Coding for Human Activity Recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 2, Article 65 (jun 2021), 26 pages. https://doi.org/10.1145/3463506

Digital Library

[23]

Harish Haresamudram, Irfan Essa, and Thomas Pl�tz. 2022. Assessing the state of self-supervised human activity recognition using wearables. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--47.

Digital Library

[24]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[25]

Rong Hu, Ling Chen, Shenghuan Miao, and Xing Tang. 2023. Swl-adapt: An unsupervised domain adaptation model with sample weight learning for cross-user wearable human activity recognition. In Proceedings of the AAAI Conference on artificial intelligence, Vol. 37. 6012--6020.

Digital Library

[26]

Sozo Inoue, Paula Lago, Tahera Hossain, Tittaya Mairittha, and Nattaya Mairittha. 2019. Integrating Activity Recognition and Nursing Care Records: The System, Deployment, and a Verification Study. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 3, Article 86 (sep 2019), 24 pages. https://doi.org/10.1145/3351244

Digital Library

[27]

Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. 2019. Deep learning for time series classification: a review. Data mining and knowledge discovery 33, 4 (2019), 917--963.

Digital Library

[28]

Aryan Jadon, Avinash Patil, and Shruti Jadon. 2022. A Comprehensive Survey of Regression Based Loss Functions for Time Series Forecasting. arXiv preprint arXiv:2211.02989 (2022).

[29]

Yash Jain, Chi Ian Tang, Chulhong Min, Fahim Kawsar, and Akhil Mathur. 2022. Collossl: Collaborative self-supervised learning for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1--28.

Digital Library

[30]

Jeya Vikranth Jeyakumar, Ankur Sarker, Luis Antonio Garcia, and Mani Srivastava. 2023. X-CHAR: A Concept-Based Explainable Complex Human Activity Recognition Model. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 7, 1, Article 17 (mar 2023), 28 pages. https://doi.org/10.1145/3580804

Digital Library

[31]

Junguang Jiang, Yang Shu, Jianmin Wang, and Mingsheng Long. 2022. Transferability in deep learning: A survey. arXiv preprint arXiv:2201.05867 (2022).

[32]

Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, Vol. 1. 2.

[33]

Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. 2022. Transformers in vision: A survey. ACM computing surveys (CSUR) 54, 10s (2022), 1--41.

Digital Library

[34]

Donghyun Kim, Kaihong Wang, Stan Sclaroff, and Kate Saenko. 2022. A broad study of pre-training for domain generalization and adaptation. In European Conference on Computer Vision. Springer, 621--638.

Digital Library

[35]

Daehee Kim, Youngjun Yoo, Seunghyun Park, Jinkyu Kim, and Jaekoo Lee. 2021. Selfreg: Self-supervised contrastive regularization for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9619--9628.

[36]

Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. 2021. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations.

[37]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[38]

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023. Segment anything. arXiv preprint arXiv:2304.02643 (2023).

[39]

Hyeokhyen Kwon, Catherine Tong, Harish Haresamudram, Yan Gao, Gregory D. Abowd, Nicholas D. Lane, and Thomas Pl�tz. 2020. IMUTube: Automatic Extraction of Virtual on-Body Accelerometry from Video for Human Activity Recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 3, Article 87 (sep 2020), 29 pages. https://doi.org/10.1145/3411841

Digital Library

[40]

Zikang Leng, Hyeokhyen Kwon, and Thomas Pl�tz. 2023. On the Benefit of Generative Foundation Models for Human Activity Recognition. arXiv preprint arXiv:2310.12085 (2023).

[41]

Youpeng Li, Xuyu Wang, and Lingling An. 2023. Hierarchical Clustering-Based Personalized Federated Learning for Robust and Fair Human Activity Recognition. 7, 1, Article 20 (mar 2023), 38 pages. https://doi.org/10.1145/3580795

Digital Library

[42]

Wang Lu, Jindong Wang, Yiqiang Chen, Sinno Jialin Pan, Chunyu Hu, and Xin Qin. 2022. Semantic-discriminative mixup for generalizable sensor-based cross-domain activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 2 (2022), 1--19.

Digital Library

[43]

Wang Lu, Jindong Wang, Xinwei Sun, Yiqiang Chen, and Xing Xie. 2023. Out-of-distribution Representation Learning for Time Series Classification. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=gUZWOE42l6Q

[44]

Qianli Ma, Zhen Liu, Zhenjing Zheng, Ziyang Huang, Siying Zhu, Zhongzhong Yu, and James T Kwok. 2023. A Survey on Time-Series Pre-Trained Models. arXiv preprint arXiv:2305.10716 (2023).

[45]

Nattaya Mairittha, Tittaya Mairittha, Paula Lago, and Sozo Inoue. 2021. CrowdAct: Achieving High-Quality Crowdsourced Datasets in Mobile Activity Recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 1, Article 50 (mar 2021), 32 pages. https://doi.org/10.1145/3432222

Digital Library

[46]

Mohammad Malekzadeh, Richard G Clegg, Andrea Cavallaro, and Hamed Haddadi. 2019. Mobile sensor data anonymization. In Proceedings of the international conference on internet of things design and implementation. 49--58.

Digital Library

[47]

Akhil Mathur, Anton Isopoussu, Nadia Berthouze, Nicholas D. Lane, and Fahim Kawsar. 2019. Unsupervised Domain Adaptation for Robust Sensory Systems. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers (London, United Kingdom) (UbiComp/ISWC '19 Adjunct). Association for Computing Machinery, New York, NY, USA, 505--509. https://doi.org/10.1145/3341162.3345609

Digital Library

[48]

Alan Mazankiewicz, Klemens B�hm, and Mario Berges. 2020. Incremental Real-Time Personalization in Human Activity Recognition Using Domain Adaptive Batch Normalization. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 4, Article 144 (dec 2020), 20 pages. https://doi.org/10.1145/3432230

Digital Library

[49]

Riccardo Presotto, Sannara Ek, Gabriele Civitarese, Fran�ois Portet, Philippe Lalanda, and Claudio Bettini. 2023. Combining Public Human Activity Recognition Datasets to Mitigate Labeled Data Scarcity. arXiv preprint arXiv:2306.13735 (2023).

[50]

Hangwei Qian, Sinno Jialin Pan, and Chunyan Miao. 2021. Latent independent excitation for generalizable sensor-based cross-person activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 11921--11929.

[51]

Xin Qin, Yiqiang Chen, Jindong Wang, and Chaohui Yu. 2019. Cross-dataset activity recognition via adaptive spatial-temporal transfer learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 4 (2019), 1--25.

Digital Library

[52]

Xin Qin, Jindong Wang, Yiqiang Chen, Wang Lu, and Xinlong Jiang. 2022. Domain generalization for activity recognition via adaptive feature fusion. ACM Transactions on Intelligent Systems and Technology 14, 1 (2022), 1--21.

Digital Library

[53]

Xin Qin, Jindong Wang, Shuo Ma, Wang Lu, Yongchun Zhu, Xing Xie, and Yiqiang Chen. 2023. Generalizable Low-Resource Activity Recognition with Diverse and Discriminative Representation Learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Long Beach, CA, USA) (KDD '23). Association for Computing Machinery, New York, NY, USA, 1943--1953. https://doi.org/10.1145/3580305.3599360

Digital Library

[54]

Xia Qingxin, Atsushi Wada, Joseph Korpela, Takuya Maekawa, and Yasuo Namioka. 2019. Unsupervised factory activity recognition with wearable sensors using process instruction information. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 1--23.

Digital Library

[55]

Valentin Radu and Maximilian Henne. 2019. Vision2sensor: Knowledge transfer across sensing modalities for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--21.

Digital Library

[56]

Jorge-L Reyes-Ortiz, Luca Oneto, Albert Sam�, Xavier Parra, and Davide Anguita. 2016. Transition-aware human activity recognition using smartphones. Neurocomputing 171 (2016), 754--767.

Digital Library

[57]

Seyed Ali Rokni, Marjan Nourollahi, and Hassan Ghasemzadeh. 2018. Personalized human activity recognition using convolutional neural networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.

[58]

Aaqib Saeed, Tanir Ozcelebi, and Johan Lukkien. 2019. Multi-task self-supervised learning for human activity detection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 1--30.

Digital Library

[59]

Sadiq Sani, Nirmalie Wiratunga, Stewart Massie, and Kay Cooper. 2017. kNN sampling for personalised human activity recognition. In Case-Based Reasoning Research and Development: 25th International Conference, ICCBR 2017, Trondheim, Norway, June 26-28, 2017, Proceedings 25. Springer, 330--344.

[60]

Panneer Selvam Santhalingam, Parth Pathak, Huzefa Rangwala, and Jana Kosecka. 2023. Synthetic Smartwatch IMU Data Generation from In-the-wild ASL Videos. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 2 (2023), 1--34.

Digital Library

[61]

Shuai Shao, Yu Guan, Bing Zhai, Paolo Missier, and Thomas Pl�tz. 2023. ConvBoost: Boosting ConvNets for Sensor-Based Activity Recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 7, 2, Article 75 (jun 2023), 21 pages. https://doi.org/10.1145/3596234

Digital Library

[62]

Taoran Sheng and Manfred Huber. 2020. Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 2, Article 57 (jun 2020), 18 pages. https://doi.org/10.1145/3397330

Digital Library

[63]

Muhammad Shoaib, Stephan Bosch, Ozlem Durmaz Incel, Hans Scholten, and Paul JM Havinga. 2014. Fusion of smartphone motion sensors for physical activity recognition. Sensors 14, 6 (2014), 10146--10176.

[64]

Sima Siami-Namini, Neda Tavakoli, and Akbar Siami Namin. 2019. The performance of LSTM and BiLSTM in forecasting time series. In 2019 IEEE International conference on big data (Big Data). IEEE, 3285--3292.

[65]

Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kj�rgaard, Anind Dey, Tobias Sonne, and Mads M�ller Jensen. 2015. Smart Devices Are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems (Seoul, South Korea) (SenSys '15). Association for Computing Machinery, New York, NY, USA, 127--140. https://doi.org/10.1145/2809695.2809718

Digital Library

[66]

Jie Su, Zhenyu Wen, Tao Lin, and Yu Guan. 2022. Learning disentangled behaviour patterns for wearable-based human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1--19.

Digital Library

[67]

Qingyu Tan, Ruidan He, Lidong Bing, and Hwee Tou Ng. 2022. Domain Generalization for Text Classification with Memory-Based Supervised Contrastive Learning. In Proceedings of the 29th International Conference on Computational Linguistics. 6916--6926.

[68]

Chi Ian Tang, Ignacio Perez-Pozuelo, Dimitris Spathis, Soren Brage, Nick Wareham, and Cecilia Mascolo. 2021. Selfhar: Improving human activity recognition through self-training with unlabeled data. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 5, 1 (2021), 1--30.

Digital Library

[69]

Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 (2022).

[70]

Xue Wang Liang Sun Rong Jin Tian Zhou, Peisong Niu. 2023. One Fits All: Power General Time Series Analysis by Pretrained LM. In NeurIPS.

[71]

Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. 2021. Mlp-mixer: An all-mlp architecture for vision. Advances in neural information processing systems 34 (2021), 24261--24272.

[72]

Catherine Tong, Jinchen Ge, and Nicholas D. Lane. 2022. Zero-Shot Learning for IMU-Based Activity Recognition Using Video Embeddings. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 4, Article 180 (dec 2022), 23 pages. https://doi.org/10.1145/3494995

Digital Library

[73]

Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2017. Instance Normalization: The Missing Ingredient for Fast Stylization. arXiv:1607.08022 [cs.CV]

[74]

Terry T. Um, Franz M. J. Pfister, Daniel Pichler, Satoshi Endo, Muriel Lang, Sandra Hirche, Urban Fietzek, and Dana Kulić. 2017. Data Augmentation of Wearable Sensor Data for Parkinson's Disease Monitoring Using Convolutional Neural Networks. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (ICMI 2017). ACM, New York, NY, USA, 216--220. https://doi.org/10.1145/3136755.3136817

Digital Library

[75]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).

[76]

Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Philip Yu. 2022. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering (2022).

[77]

Jindong Wang, Vincent W Zheng, Yiqiang Chen, and Meiyu Huang. 2018. Deep transfer learning for cross-domain activity recognition. In proceedings of the 3rd International Conference on Crowd Science and Engineering. 1--8.

Digital Library

[78]

Zhiguang Wang, Weizhong Yan, and Tim Oates. 2017. Time series classification from scratch with deep neural networks: A strong baseline. In 2017 International joint conference on neural networks (IJCNN). IEEE, 1578--1585.

[79]

Qingsong Wen, Liang Sun, Fan Yang, Xiaomin Song, Jingkun Gao, Xue Wang, and Huan Xu. 2020. Time series data augmentation for deep learning: A survey. arXiv preprint arXiv:2002.12478 (2020).

[80]

Garrett Wilson and Diane J Cook. 2020. A survey of unsupervised deep domain adaptation. ACM Transactions on Intelligent Systems and Technology (TIST) 11, 5 (2020), 1--46.

Digital Library

[81]

Qingxin Xia, Joseph Korpela, Yasuo Namioka, and Takuya Maekawa. 2020. Robust Unsupervised Factory Activity Recognition with Body-Worn Accelerometer Using Temporal Structure of Multiple Sensor Data Motifs. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 3, Article 97 (sep 2020), 30 pages. https://doi.org/10.1145/3411836

Digital Library

[82]

Huatao Xu, Pengfei Zhou, Rui Tan, and Mo Li. 2023. Practically Adopting Human Activity Recognition. In Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. 1--15.

Digital Library

[83]

Huatao Xu, Pengfei Zhou, Rui Tan, Mo Li, and Guobin Shen. 2021. LIMU-BERT: Unleashing the Potential of Unlabeled Data for IMU Sensing Applications. In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems (Coimbra, Portugal) (SenSys '21). Association for Computing Machinery, New York, NY, USA, 220--233. https://doi.org/10.1145/3485730.3485937

Digital Library

[84]

Xuhai Xu, Xin Liu, Han Zhang, Weichen Wang, Subigya Nepal, Yasaman Sefidgar, Woosuk Seo, Kevin S. Kuehn, Jeremy F. Huckins, Margaret E. Morris, Paula S. Nurius, Eve A. Riskin, Shwetak Patel, Tim Althoff, Andrew Campbell, Anind K. Dey, and Jennifer Mankoff. 2023. GLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 4, Article 190 (jan 2023), 34 pages. https://doi.org/10.1145/3569485

Digital Library

[85]

Shuochao Yao, Shaohan Hu, Yiran Zhao, Aston Zhang, and Tarek Abdelzaher. 2017. Deepsense: A unified deep learning framework for time-series mobile sensing data processing. In Proceedings of the 26th international conference on world wide web. 351--360.

Digital Library

[86]

Xuanke You, Lan Zhang, Haikuo Yu, Mu Yuan, and Xiang-Yang Li. 2022. KATN: Key Activity Detection via Inexact Supervised Learning. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 4, Article 189 (dec 2022), 26 pages. https://doi.org/10.1145/3494957

Digital Library

[87]

Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. 2017. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017).

[88]

Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. 2022. Self-supervised contrastive pre-training for time series via time-frequency consistency. Advances in Neural Information Processing Systems 35 (2022), 3988--4003.

[89]

Yi-Fan Zhang, Jindong Wang, Jian Liang, Zhang Zhang, Baosheng Yu, Liang Wang, Dacheng Tao, and Xing Xie. 2023. Domain-Specific Risk Minimization for Domain Generalization. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Long Beach, CA, USA) (KDD '23). Association for Computing Machinery, New York, NY, USA, 3409--3421. https://doi.org/10.1145/3580305.3599313

Digital Library

[90]

Zhilu Zhang and Mert R. Sabuncu. 2018. Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (Montr�al, Canada) (NIPS'18). Curran Associates Inc., Red Hook, NY, USA, 8792--8802.

[91]

Si Zuo, Vitor Fortes, Sungho Suh, Stephan Sigg, and Paul Lukowicz. 2023. Unsupervised Diffusion Model for Sensor-Based Human Activity Recognition. In Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing (Cancun, Quintana Roo, Mexico) (UbiComp/ISWC '23 Adjunct). Association for Computing Machinery, New York, NY, USA, 205. https://doi.org/10.1145/3594739.3610797

Digital Library

Cited By

Salunkhe SShinde SPatil PBurchard RKostakos VKay JHoang T(2024)Augmentation Appproaches to Refine Wearable Human Activity RecognitionCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678464(603-606)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678464
Hopp MHartleb HBurchard RKostakos VKay JHoang T(2024)TA-DA! - Improving Activity Recognition using Temporal Adapters and Data AugmentationCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678454(551-554)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678454
Wang HWang SLin LYang YWang SWen HSerra ESpezzano F(2024)Behavior-aware Sparse Trajectory Recovery in Last-mile Delivery with Multi-scale Attention FusionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3680079(4931-4938)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3680079
Show More Cited By

Index Terms

CrossHAR: Generalizing Cross-dataset Human Activity Recognition via Hierarchical Self-Supervised Pretraining
1. Human-centered computing
  1. Ubiquitous and mobile computing

Recommendations

Self-supervised learning with randomized cross-sensor masked reconstruction for human activity recognition
Abstract
Self-supervised learning (SSL) has gained prominence in the field of accelerometer-based human activity recognition (HAR) due to its ability to learn from both labeled and unlabeled data. While labeled data acquisition is costly, it is relatively ...
Graphical abstract

Display Omitted
Highlights
- New self-supervised cross-sensor learning auxiliary task (RCSMR) for HAR.
- Large-scale dual-sensor pre-training, single-sensor downstream training.
- RCSMR pre-trained Transformer outperforms supervised & SSL methods on 7 HAR ...
SelfPAB: large-scale pre-training on accelerometer data for human activity recognition
Abstract
Annotating accelerometer-based physical activity data remains a challenging task, limiting the creation of robust supervised machine learning models due to the scarcity of large, labeled, free-living human activity recognition (HAR) datasets. ...
Efficient human activity recognition: A deep convolutional transformer-based contrastive self-supervised approach using wearable sensors
Abstract
Artificial intelligence has advanced the applications of sensor-based human motion capture and recognition technology in various engineering fields, such as human–robot collaboration and health monitoring. Deep learning methods can achieve ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 8, Issue 2

May 2024

1330 pages

EISSN:2474-9567

DOI:10.1145/3665317

Issue’s Table of Contents

Copyright � 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 May 2024

Published in�IMWUT�Volume 8, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science and Technology Major Project

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
1,017
Total Downloads

Downloads (Last 12 months)1,017
Downloads (Last 6 weeks)252

Reflects downloads up to 22 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Salunkhe SShinde SPatil PBurchard RKostakos VKay JHoang T(2024)Augmentation Appproaches to Refine Wearable Human Activity RecognitionCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678464(603-606)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678464
Hopp MHartleb HBurchard RKostakos VKay JHoang T(2024)TA-DA! - Improving Activity Recognition using Temporal Adapters and Data AugmentationCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678454(551-554)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678454
Wang HWang SLin LYang YWang SWen HSerra ESpezzano F(2024)Behavior-aware Sparse Trajectory Recovery in Last-mile Delivery with Multi-scale Attention FusionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3680079(4931-4938)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3680079
Yang GTu MLi ZHang JLiu TLiu RDing YYang YZhang DSerra ESpezzano F(2024)Behavior-Aware Hypergraph Convolutional Network for Illegal Parking Prediction with Multi-Source Contextual InformationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679563(2827-2836)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679563

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents