skip to main content
research-article
Open access

The Effects of Network Outages on User Experience in Augmented Reality Based Remote Collaboration - An Empirical Study

Published: 18 October 2021 Publication History

Abstract

Augmented Reality (AR) applications can enable geographically distant users to collaborate using shared video feeds or interactive 3D holograms, and may be particularly useful in the socially distant context of the Covid-19 pandemic. However, a good user experience is key for their success and could be negatively impacted by network impairments, which are an inevitable occurrence in today's best-effort Internet. In this paper, we present the findings of an empirical user study, aimed at understanding the effects of network outages, on user experience and behavior, in a collaborative AR task. We highlight how network outages affected users in different ways depending on their role in the collaborative task, and how giving users explicit information about poor network conditions helped them deal with some of these negative effects. Furthermore, we report the strategies that users themselves adopted, to deal with outages, such as batching instructions, or shifting to a different spatial referencing style when communicating with their partners. Lastly, based on our findings, we present some design implications for future remote-collaborative AR applications.

References

[1]
[n. d.]. http://arsharing.cs.washington.edu/index.html
[2]
[n. d.]. https://objecttheory.com/platform/
[3]
2020. 10 BEST Augmented Reality Glasses (Smart Glasses) In 2021. https://www.softwaretestinghelp.com/best-augmented-reality-glasses/
[4]
Giuseppe Aceto, Alessio Botta, Pietro Marchetta, Valerio Persico, and Antonio Pescap�. 2018. A comprehensive survey on internet outages. Journal of Network and Computer Applications, Vol. 113 (2018), 36--63.
[5]
Sudhir Aggarwal, Hemant Banavar, Amit Khandelwal, Sarit Mukherjee, and Sampath Rangarajan. 2004. Accuracy in dead-reckoning based distributed multi-player games. In Proceedings of 3rd ACM SIGCOMM workshop on Network and system support for games. 161--165.
[6]
Tooba Ahsen, Fahad R Dogar, and Aaron L Gardony. 2019. Exploring the Impact of Network Impairments on Remote Collaborative Augmented Reality Applications. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, LBW2619.
[7]
Deepak Akkil and Poika Isokoski. 2016a. Accuracy of interpreting pointing gestures in egocentric view. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 262--273.
[8]
Deepak Akkil and Poika Isokoski. 2016b. Gaze Augmentation in Egocentric Video Improves Awareness of Intention. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1573--1584.
[9]
Deepak Akkil, Jobin Mathew James, Poika Isokoski, and Jari Kangas. 2016. GazeTorch: Enabling Gaze Awareness in Collaborative Physical Tasks. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1151--1158.
[10]
Judith Amores, Xavier Benavides, and Pattie Maes. 2015. Showme: A remote collaboration system that supports immersive gestural communication. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. 1343--1348.
[11]
Huidong Bai, Prasanth Sasikumar, Jing Yang, and Mark Billinghurst. 2020. A user study on mixed reality remote collaboration with eye gaze and hand gesture sharing. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1--13.
[12]
Yahn W Bernier. 2001. Latency compensating methods in client/server in-game protocol design and optimization. In Game Developers Conference, Vol. 98033.
[13]
Henrik Brun, Robin Anton Birkeland Bugge, LKR Suther, Sigurd Birkeland, Rahul Kumar, Egidijus Pelanis, and Ole Jacob Elle. 2019. Mixed reality holograms for heart surgery planning: first user experience in congenital heart disease. European Heart Journal-Cardiovascular Imaging, Vol. 20, 8 (2019), 883--888.
[14]
Wolfgang B�schel. [n. d.]. Challenges in Collaborative Immersive Visualization. ([n. d.]).
[15]
Simon Butscher, Sebastian Hubenschmid, Jens M�ller, Johannes Fuchs, and Harald Reiterer. 2018. Clusters, trends, and outliers: How immersive technologies can facilitate the collaborative analysis of multidimensional data. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--12.
[16]
Tolga K Capin and Igor S Pandzic. 1997. A dead-reckoning algorithm for virtual human figures. In Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality. IEEE, 161--169.
[17]
Armando L Caro, Janardhan R Iyengar, Paul D Amer, Sourabh Ladha, Gerard J Heinz, and Keyur C Shah. 2003. SCTP: a proposed standard for robust internet data transport. Computer, Vol. 36, 11 (2003), 56--63.
[18]
Kuan-Ta Chen, Chun-Ying Huang, Polly Huang, and Chin-Laung Lei. 2006. Quantifying Skype user satisfaction. ACM SIGCOMM Computer Communication Review, Vol. 36, 4 (2006), 399--410.
[19]
Marshini Chetty, Srikanth Sundaresan, Sachit Muckaden, Nick Feamster, and Enrico Calandro. 2013. Measuring broadband performance in South Africa. In Proceedings of the 4th Annual Symposium on Computing for Development. ACM, 1.
[20]
Herbert H. Clark. 1996. Common ground .Cambridge University Press, 92--122. https://doi.org/10.1017/CBO9780511620539.005
[21]
Herbert H Clark and Susan E Brennan. 1991. Grounding in communication. (1991).
[22]
Catherine Durnell Cramton. 2001. The mutual knowledge problem and its consequences for dispersed collaboration. Organization science, Vol. 12, 3 (2001), 346--371.
[23]
Alberto Dainotti, Claudio Squarcella, Emile Aben, Kimberly C Claffy, Marco Chiesa, Michele Russo, and Antonio Pescap�. 2011. Analysis of country-wide internet outages caused by censorship. In Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference. ACM, 1--18.
[24]
Barrett Ens, Sarah Goodwin, Arnaud Prouzeau, Fraser Anderson, Florence Y Wang, Samuel Gratzl, Zac Lucarelli, Brendan Moyle, Jim Smiley, and Tim Dwyer. 2020. Uplift: A Tangible and Immersive Tabletop System for Casual Collaborative Visual Analytics. IEEE Transactions on Visualization and Computer Graphics (2020).
[25]
Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, and Mark Billinghurst. 2019. Revisiting collaboration through mixed reality: The evolution of groupware. International Journal of Human-Computer Studies, Vol. 131 (2019), 81--98.
[26]
Brian Eriksson, Ramakrishnan Durairajan, and Paul Barford. 2013. Riskroute: a framework for mitigating network outage threats. In Proceedings of the ninth ACM conference on Emerging networking experiments and technologies. ACM, 405--416.
[27]
P Esteban, JE Jaramillo, N �lvarez, J Restrepo, and H Trefftz. 2004. Augmented Reality: a space for the understanding of multi-variate Calculus. In Proceedings of the IADATE conference. Spain.
[28]
Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias H�llerer. 2014a. In touch with the remote world: Remote collaboration with augmented reality drawings and virtual navigation. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology. 197--205.
[29]
Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias H�llerer. 2014b. World-stabilized annotations and virtual scene navigation for remote collaboration. In Proceedings of the 27th annual ACM symposium on User interface software and technology. 449--459.
[30]
Aaron M Genest, Carl Gutwin, Anthony Tang, Michael Kalyn, and Zenja Ivkovic. 2013. KinectArms: a toolkit for capturing and displaying arm embodiments in distributed tabletop groupware. In Proceedings of the 2013 conference on Computer supported cooperative work. 157--166.
[31]
Darren Gergle, Robert E Kraut, and Susan R Fussell. 2004 a. Action as language in a shared visual space. In Proceedings of the 2004 ACM conference on Computer supported cooperative work. ACM, 487--496.
[32]
Darren Gergle, Robert E Kraut, and Susan R Fussell. 2004 b. Language efficiency and visual technology: Minimizing collaborative effort with visual information. Journal of language and social psychology, Vol. 23, 4 (2004), 491--517.
[33]
Darren Gergle, Robert E Kraut, and Susan R Fussell. 2013. Using visual information for grounding and awareness in collaborative tasks. Human-Computer Interaction, Vol. 28, 1 (2013), 1--39.
[34]
Rapha�l Grasset, Philip Lamb, and Mark Billinghurst. 2005. Evaluation of mixed-space collaboration. In Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society, 90--99.
[35]
Sebastian G�nther, Sven Kratz, Daniel Avrahami, and Max M�hlh"auser. 2018. Exploring Audio, Visual, and Tactile Cues for Synchronous Remote Assistance. In Proceedings of the 11th Pervasive Technologies Related to Assistive Environments Conference. 339--344.
[36]
Felix G Hamza-Lup, Jannick P Rolland, and Charles Hughes. 2018. A distributed augmented reality system for medical training and simulation. arXiv preprint arXiv:1811.12815 (2018).
[37]
Osama Haq and Fahad R Dogar. 2015. Leveraging the power of cloud for reliable wide area communication. In Proc. ACM HotNets.
[38]
Tomoki Itamiya, Toshinori Iwai, and Tsuyoshi Kaneko. 2018. The Holographic Human for surgical navigation using Microsoft HoloLens. EPiC Series in Engineering, Vol. 1 (2018), 26--30.
[39]
Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, and Jan Kautz. 2018. Super slomo: High quality estimation of multiple intermediate frames for video interpolation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9000--9008.
[40]
Junchen Jiang, Rajdeep Das, Ganesh Ananthanarayanan, Philip A Chou, Venkata Padmanabhan, Vyas Sekar, Esbjorn Dominique, Marcin Goliszewski, Dalibor Kukoleca, Renat Vafin, et al. 2016. Via: Improving internet telephony call quality using predictive relay selection. In Proc. SIGCOMM'16. ACM.
[41]
Wenyu Jiang and Henning Schulzrinne. 2003. Assessment of voip service availability in the current internet. In Proceedings of the 4th International Workshop on Passive and Active Network Measurement (PAM 2003).
[42]
Laura Casado Joey Hadden. 2020. 21 major companies that have announced employees can work remotely long-term. https://www.businessinsider.com/companies-asking-employees-to-work-from-home-due-to-coronavirus-2020
[43]
Steven Johnson, Madeleine Gibson, and Bilge Mutlu. 2015. Handheld or handsfree?: Remote collaboration via lightweight head-mounted displays and handheld devices. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 1825--1836.
[44]
Ethan Katz-Bassett, Colin Scott, David R Choffnes, �talo Cunha, Vytautas Valancius, Nick Feamster, Harsha V Madhyastha, Thomas Anderson, and Arvind Krishnamurthy. 2012. LIFEGUARD: Practical repair of persistent route failures. In Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication. ACM, 395--406.
[45]
Hannes Kaufmann. 2002. Construct3D: an augmented reality application for mathematics and geometry education. In Proceedings of the tenth ACM international conference on Multimedia. 656--657.
[46]
Seungwon Kim, Allison Jing, Hanhoon Park, Gun A Lee, Weidong Huang, and Mark Billinghurst. 2020. Hand-in-Air (HiA) and Hand-on-Target (HoT) Style Gesture Cues for Mixed Reality Collaboration. IEEE Access, Vol. 8 (2020), 224145--224161.
[47]
Kiyoshi Kiyokawa, Mark Billinghurst, Sohan E Hayes, Anoop Gupta, Yuki Sannohe, and Hirokazu Kato. 2002. Communication behaviors of co-located users in collaborative AR interfaces. In Proceedings of the 1st International Symposium on Mixed and Augmented Reality. IEEE Computer Society, 139.
[48]
Adam Langley, Alistair Riddoch, Alyssa Wilk, Antonio Vicente, Charles Krasic, Dan Zhang, Fan Yang, Fedor Kouranov, Ian Swett, Janardhan Iyengar, et al. 2017. The quic transport protocol: Design and internet-scale deployment. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 183--196.
[49]
Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Chapter 11 - Analyzing qualitative data. In Research Methods in Human Computer Interaction (Second Edition) second edition ed.), Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser (Eds.). Morgan Kaufmann, Boston, 299 -- 327. https://doi.org/10.1016/B978-0--12--805390--4.00011-X
[50]
Jun Li and Scott Brooks. 2011. I-seismograph: Observing and measuring Internet earthquakes. In INFOCOM, 2011 Proceedings IEEE. IEEE, 2624--2632.
[51]
Stephan Lukosch, Heide Lukosch, Dragos Datcu, and Marina Cidota. 2015. On the spot information in augmented reality for teams in the security domain. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 983--988.
[52]
Tahir Mahmood, Willis Fulmer, Neelesh Mungoli, Jian Huang, and Aidong Lu. 2019. Improving information sharing and collaborative analysis for remote geospatial visualization using mixed reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 236--247.
[53]
Shawn McKee, Mike O'Connor, Soichi Hayashi, Marian Babik, Brian Tierney, Ilija Vukotic, Les Cottrell, and Henryk Giemza. 2016. ICFA SCIC Network Monitoring Report. Technical Report.
[54]
Microsoft. [n. d.] a. Microsoft HoloLens. https://www.microsoft.com/en-us/hololens [Online; accessed 21-September-2018].
[55]
Microsoft. [n. d.] b. Mixed Reality Toolkit. https://github.com/Microsoft/MixedRealityToolkit-Unity [Online; accessed 21-September-2018].
[56]
Jens M�ller, Roman R"adle, and Harald Reiterer. 2016. Virtual Objects as Spatial Cues in Collaborative Mixed Reality Environments: How They Shape Communication Behavior and User Task Load. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1245--1249.
[57]
Jens M�ller, Roman R"adle, and Harald Reiterer. 2017. Remote collaboration with mixed reality displays: how shared virtual landmarks facilitate spatial referencing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 6481--6486.
[58]
Nahal Norouzi*, Austin Erickson*, Kangsoo Kim, Ryan Schubert, Joseph LaViola, Gerd Bruder, and Greg Welch. 2019. Effects of shared gaze parameters on visual target identification task performance in augmented reality. In Symposium on Spatial User Interaction. 1--11.
[59]
Lyn Pemberton and Marcus Winter. 2009. Collaborative augmented reality in schools. (2009).
[60]
Thammathip Piumsomboon, Arindam Dey, Barrett Ens, Gun Lee, and Mark Billinghurst. 2019 a. The effects of sharing awareness cues in collaborative mixed reality. Frontiers in Robotics and AI, Vol. 6 (2019), 5.
[61]
Thammathip Piumsomboon, Gun A Lee, Jonathon D Hart, Barrett Ens, Robert W Lindeman, Bruce H Thomas, and Mark Billinghurst. 2018. Mini-me: An adaptive avatar for mixed reality remote collaboration. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1--13.
[62]
Thammathip Piumsomboon, Gun A Lee, Andrew Irlitti, Barrett Ens, Bruce H Thomas, and Mark Billinghurst. 2019 b. On the shoulder of the giant: A multi-scale mixed reality collaboration with 360 video sharing and tangible interaction. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1--17.
[63]
Thammathip Piumsomboon, Youngho Lee, Gun Lee, and Mark Billinghurst. 2017. CoVAR: a collaborative virtual and augmented reality system for remote collaboration. In SIGGRAPH Asia 2017 Emerging Technologies. 1--2.
[64]
Ronald Poelman, Oytun Akman, Stephan Lukosch, and Pieter Jonker. 2012. As if being there: mediated reality for crime scene investigation. In Proceedings of the ACM 2012 conference on computer supported cooperative work. 1267--1276.
[65]
Lin Quan, John Heidemann, and Yuri Pradkin. 2013. Trinocular: Understanding internet reliability through adaptive probing. ACM SIGCOMM Computer Communication Review, Vol. 43, 4 (2013), 255--266.
[66]
Fraser JM Reid and Susan E Reed. 2007. Conversational grounding and visual access in collaborative design. CoDesign, Vol. 3, 2 (2007), 111--122.
[67]
Kimberly Ruth, Tadayoshi Kohno, and Franziska Roesner. 2019. Secure multi-user content sharing for augmented reality applications. In 28th USENIX Security Symposium ($$USENIX$$ Security 19). 141--158.
[68]
Harvey Sacks, Emanuel A Schegloff, and Gail Jefferson. 1978. A simplest systematics for the organization of turn taking for conversation. In Studies in the organization of conversational interaction. Elsevier, 7--55.
[69]
M.F. Schober. 1995. Speakers, addressees, and frames of reference: Whose effort is minimized in conversations about locations? Discourse Processes, Vol. 20, 2 (1995), 219--247.
[70]
Ronell Sicat, Jiabao Li, JunYoung Choi, Maxime Cordeil, Won-Ki Jeong, Benjamin Bach, and Hanspeter Pfister. 2018. Dxr: A toolkit for building immersive data visualizations. IEEE transactions on visualization and computer graphics, Vol. 25, 1 (2018), 715--725.
[71]
Rajinder S Sodhi, Brett R Jones, David Forsyth, Brian P Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 179--188.
[72]
Theophilus Teo, Louise Lawrence, Gun A Lee, Mark Billinghurst, and Matt Adcock. 2019. Mixed reality remote collaboration combining 360 video and 3d reconstruction. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1--14.
[73]
Trint. [n. d.]. Trint. https://trint.com/
[74]
Unity. [n. d.]. Unity. https://unity3d.com [Online; accessed 21-September-2018].
[75]
Yang Xu, Chenguang Yu, Jingjiang Li, and Yong Liu. 2012. Video telephony for end-consumers: measurement study of Google+, iChat, and Skype. In Proceedings of the 2012 Internet Measurement Conference. 371--384.
[76]
Naomi Yamashita, Katsuhiko Kaji, Hideaki Kuzuoka, and Keiji Hirata. 2011. Improving visibility of remote gestures in distributed tabletop collaboration. In Proceedings of the ACM 2011 conference on Computer supported cooperative work. 95--104.
[77]
Jeffrey M Zacks and Pascale Michelon. 2005. Transformations of visuospatial images. Behavioral and Cognitive Neuroscience Reviews, Vol. 4, 2 (2005), 96--118.

Cited By

View all
  • (2023)Divided at the Edge - Measuring Performance and the Digital Divide of Cloud Edge Data CentersProceedings of the ACM on Networking10.1145/36291381:CoNEXT3(1-23)Online publication date: 28-Nov-2023
  • (2023)QoE enhancement of the industrial metaverse based on Mixed Reality application optimizationDisplays10.1016/j.displa.2023.10246379(102463)Online publication date: Sep-2023
  • (2022)Understanding Multi-View Collaboration between Augmented Reality and Remote Desktop UsersProceedings of the ACM on Human-Computer Interaction10.1145/35556076:CSCW2(1-27)Online publication date: 11-Nov-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 5, Issue CSCW2
CSCW2
October 2021
5376 pages
EISSN:2573-0142
DOI:10.1145/3493286
Issue’s Table of Contents
This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 October 2021
Published in�PACMHCI�Volume 5, Issue CSCW2

Check for updates

Author Tags

  1. augmented reality
  2. common ground
  3. distributed collaboration
  4. mixed reality
  5. network impairments
  6. network outages
  7. remote collaboration
  8. shared holograms
  9. spatial referencing
  10. user experience

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)232
  • Downloads (Last 6 weeks)24
Reflects downloads up to 22 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2023)Divided at the Edge - Measuring Performance and the Digital Divide of Cloud Edge Data CentersProceedings of the ACM on Networking10.1145/36291381:CoNEXT3(1-23)Online publication date: 28-Nov-2023
  • (2023)QoE enhancement of the industrial metaverse based on Mixed Reality application optimizationDisplays10.1016/j.displa.2023.10246379(102463)Online publication date: Sep-2023
  • (2022)Understanding Multi-View Collaboration between Augmented Reality and Remote Desktop UsersProceedings of the ACM on Human-Computer Interaction10.1145/35556076:CSCW2(1-27)Online publication date: 11-Nov-2022
  • (2022)Understanding the Impact of Technical Issues on People’s Perception and Attribution of Responsibility in VideoconferencingExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519618(1-6)Online publication date: 27-Apr-2022

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media