research-article

Optimizing activity data collection with gamification points using uncertainty based active learning

Authors:

Nattaya Mairittha,

Tittaya Mairittha,

Sozo InoueAuthors Info & Claims

UbiComp/ISWC '19 Adjunct: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

Pages 761 - 767

https://doi.org/10.1145/3341162.3345585

Published: 09 September 2019 Publication History

Abstract

A big challenge for activity data collection is unavoidable to rely on users and to keep them motivated to provide labels. In this paper, we propose the idea of exploiting gamification points to motivate the users for activity data collection by using an uncertainty based active learning approach to evaluate those points. The novel idea behind this is that we approximate the score of the unlabeled examples according to the current model's uncertainty in its prediction of the corresponding activity labels, and using that score as gamification points. Thus, the users are motivated by getting gamification points as feedback based on their data annotation quality. 1,236 activity labels with smartphone sensors that we collected help to validate our proposed method. By evaluating with the dataset, the results show our proposed method has improvements in data quality, data quantity, and user engagement that reflect the improvement in activity data collection.

References

[1]

Ling Bao and Stephen S Intille. 2004. Activity recognition from user-annotated acceleration data. In International conference on pervasive computing. Springer, 1--17.

[2]

Leo Breiman. 2017. Classification and regression trees. Routledge.

[3]

Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research 16 (2002), 321--357.

[4]

Sebastian Deterding, Miguel Sicart, Lennart Nacke, Kenton O'Hara, and Dan Dixon. 2011. Gamification. using game-design elements in non-gaming contexts. In CHI'11 extended abstracts on human factors in computing systems. ACM, 2425--2428.

Digital Library

[5]

Tom Diethe, Niall Twomey, and Peter Flach. 2015. Bayesian active transfer learning in smart homes. In ICML Active Learning Workshop, Vol. 2015.

[6]

T Fawcett. 2006. An introduction to ROC analysis pattern recognition letter. (2006).

Digital Library

[7]

Zachary Fitz-Walter, Dian Tjondronegoro, and Peta Wyeth. 2011. Orientation passport: using gamification to engage university students. In Proceedings of the 23rd Australian computer-human interaction conference. ACM, 122--125.

Digital Library

[8]

Zachary Fitz-Walter and Dian W Tjondronegoro. 2011. Exploring the opportunities and challenges of using mobile sensing for gamification and achievements. In UbiComp 11: Proceedings of the 2011 ACM Conference on Ubiquitous Computing. ACM Press, 1--5.

[9]

Juho Hamari, Jonna Koivisto, Harri Sarsa, et al. 2014. Does Gamification Work?- A Literature Review of Empirical Studies on Gamification. In HICSS, Vol. 14. 3025--3034.

Digital Library

[10]

Yu-chen Ho, Ching-hu Lu, I-han Chen, Shih-shinh Huang, Ching-yao Wang, Li-chen Fu, et al. 2009. Active-learning assisted self-reconfigurable activity recognition in a dynamic environment. In Proceedings of the 2009 IEEE international conference on Robotics and Automation. IEEE Press, 1567--1572.

Digital Library

[11]

HM Sajjad Hossain, Md Abdullah Al Hafiz Khan, and Nirmalya Roy. 2017. Active learning enabled activity recognition. Pervasive and Mobile Computing 38 (2017), 312--330.

[12]

Bartosz Krawczyk. 2016. Learning from imbalanced data: open challenges and future directions. Progress in Artificial Intelligence 5, 4 (2016), 221--232.

[13]

Janette Lehmann, Mounia Lalmas, Elad Yom-Tov, and Georges Dupret. 2012. Models of user engagement. In International Conference on User Modeling, Adaptation, and Personalization. Springer, 164--175.

Digital Library

[14]

Zakkoyya H Lewis, Maria C Swartz, and Elizabeth J Lyons. 2016. What's the point?: a review of reward systems implemented in gamification interventions. Games for health journal 5, 2 (2016), 93--99.

[15]

Rong Liu, Ting Chen, and Lu Huang. 2010. Research on human activity recognition based on active learning. In 2010 International Conference on Machine Learning and Cybernetics, Vol. 1. IEEE, 285--290.

[16]

Nattaya Mairittha, Tittaya Mairittha, and Sozo Inoue. 2018. A Mobile App for Nursing Activity Recognition. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computingand Wearable Computers. ACM, 400--403.

Digital Library

[17]

Laurence Moroney. 2017. Google Analytics for Firebase. In The Definitive Guide to Firebase. Springer, 251--270.

[18]

Martin Pielot, Karen Church, and Rodrigo De Oliveira. 2014. An in-situ study of mobile phone notifications. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services. ACM, 233--242.

Digital Library

[19]

Burr Settles. 2012. Active learning. Synthesis Lectures on Artificial Intelligence and Machine Learning 6, 1 (2012), 1--114.

Digital Library

[20]

Maja Stikic, Kristof Van Laerhoven, and Bernt Schiele. 2008. Exploring semi-supervised and active learning for activity recognition. In 2008 12th IEEE International Symposium on Wearable Computers. IEEE, 81--88.

Digital Library

[21]

Francisco J Valverde-Albacete and Carmen Pel�ez-Moreno. 2014. 100% classification accuracy considered harmful: The normalized information transfer factor explains the accuracy paradox. PloS one 9, 1 (2014), e84217.

[22]

Niels Van Berkel, Jorge Goncalves, Simo Hosio, and Vassilis Kostakos. 2017. Gamification of mobile experience sampling improves data quality and quantity. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 107.

Digital Library

Cited By

Kianpisheh MMariakakis ATruong K(2024)exHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435008:1(1-30)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643500
Presotto RCivitarese GBettini C(2023)Is Querying Users Acceptable for Human Activity Recognition Based on Active Learning?2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops56833.2023.10150407(521-526)Online publication date: 13-Mar-2023
https://doi.org/10.1109/PerComWorkshops56833.2023.10150407
Uhm JKim SLee H(2022)Stimulating Suspense in Gamified Virtual Reality Sports: Effect on Flow, Fun, and Behavioral IntentionInternational Journal of Human–Computer Interaction10.1080/10447318.2022.210778239:19(3846-3858)Online publication date: 15-Aug-2022
https://doi.org/10.1080/10447318.2022.2107782
Show More Cited By

Index Terms

Optimizing activity data collection with gamification points using uncertainty based active learning
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Tracking
2. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing design and evaluation methods

Recommendations

Predicting Performance Improvement of Human Activity Recognition Model by Additional Data Collection

The development of a machine-learning-based human activity recognition (HAR) system using body-worn sensors is mainly composed of three phases: data collection, model training, and evaluation. During data collection, the HAR developer collects labeled ...
Age-based garbage collection

Modern generational garbage collectors look for garbage among the young objects, because they have high mortality; however, these objects include the very youngest objects, which clearly are still live. We introduce new garbage collection algorithms, ...
AusPlots Rangelands field data collection and publication

The TERN AusPlots Rangelands field data collection system has been developed to facilitate simple and efficient data collection by ecologists operating in the Australian outback. The infrastructure provides tooling for 'clean' data collection on mobile (...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp/ISWC '19 Adjunct: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

September 2019

1234 pages

ISBN:9781450368698

DOI:10.1145/3341162

General Chairs:
Robert Harle
Cambridge University
,
Katayoun Farrahi
University Of Southampton
,
Nicholas Lane
University Of Oxford And Samsung Ai

Copyright � 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UbiComp '19

Sponsor:

UbiComp '19: The 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 9 - 13, 2019

London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
288
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)1

Reflects downloads up to 22 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kianpisheh MMariakakis ATruong K(2024)exHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435008:1(1-30)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643500
Presotto RCivitarese GBettini C(2023)Is Querying Users Acceptable for Human Activity Recognition Based on Active Learning?2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops56833.2023.10150407(521-526)Online publication date: 13-Mar-2023
https://doi.org/10.1109/PerComWorkshops56833.2023.10150407
Uhm JKim SLee H(2022)Stimulating Suspense in Gamified Virtual Reality Sports: Effect on Flow, Fun, and Behavioral IntentionInternational Journal of Human–Computer Interaction10.1080/10447318.2022.210778239:19(3846-3858)Online publication date: 15-Aug-2022
https://doi.org/10.1080/10447318.2022.2107782
Mairittha NMairittha TLago PInoue S(2021)CrowdActProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34322225:1(1-32)Online publication date: 30-Mar-2021
https://dl.acm.org/doi/10.1145/3432222
Mairittha NMairittha TInoue S(2020)On-Device Deep Personalization for Robust Activity Data CollectionSensors10.3390/s2101004121:1(41)Online publication date: 23-Dec-2020
https://doi.org/10.3390/s21010041

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents