research-article

Understanding People's Mental Models of Mid-Air Interaction for Virtual Assembly and Shape Modeling

Authors:

Dieter W. Fellner,

Alexei SourinAuthors Info & Claims

CASA '16: Proceedings of the 29th International Conference on Computer Animation and Social Agents

Pages 139 - 146

https://doi.org/10.1145/2915926.2919330

Published: 23 May 2016 Publication History

Abstract

Naturalness of the mid-air interaction interface for virtual assembly and shape modeling is important. In order to design an interface perceived as "natural" by most people, common behaviors and mental patterns for mid-air interaction of people have to be recognized, which is an area merely explored yet. This paper serves this purpose of understanding the users' mental interaction models, in order to provide standards and recommendation for devising a natural virtual interaction interface. We tested three kinds of tasks --- manipulating tasks, deforming tasks and tool-based operating tasks on 16 participants. We have found that: 1) different features of mental models were observed for different types of tasks. Interaction techniques should be designed to match these features; 2) virtual hand self-avatar helps estimate size of virtual objects, as well as helps plan and visualize the complex process and procedures of a task, which is especially helpful for tool-based tasks; 3) bimanual interaction is witnessed as a dominant interaction mode preferred by the majority; 4) natural gestures for deforming tasks always reflect forces exerted. These suggestions are useful for designing a midair interaction interface matching users' mental models.

References

[1]

Cui, J. and Sourin, A. 2014. Feasibility Study on Free Hand Geometric Modelling Using Leap Motion in VRML/X3D. In Proceedings of the 2014 International Conference on Cyberworlds (CW '14). IEEE Computer Society, Washington, DC, USA, 389--392. DOI= http://dx.doi.org/10.1109/CW.2014.60.

Digital Library

[2]

Fuge, M., Yumer, M.E., Orbay, G. and Kara, L.B. 2012. Conceptual design and modification of freeform surfaces using dual shape representations in augmented reality environments. Computer Aided Design. 44, 10 (October 2012), 1020--1032. DOI= http://dx.doi.org/10.1016/j.cad.2011.05.009.

Digital Library

[3]

Good, M.D., Whiteside, J.A., Wixon, D.R. and Jones, S.J. 1984. Building a user-derived interface. Commun. ACM, New York, NY, USA, 27, 10 (October 1984), 1032--1043. DOI= http://dx.doi.org/10.1145/358274.3582.

Digital Library

[4]

Grandhi, S.A., Joue, G. and Mittelberg, I. 2011. Understanding naturalness and intuitiveness in gesture production: insights for touchless gestural interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 821--824. DOI= http://dx.doi.org/10.1145/1978942.1979061.

Digital Library

[5]

Guiard, Y. 1987. Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of motor behavior, 19, 4 (1987), 486--517.

[6]

Kang, J., Zhong, K., Qin, S., Wang, H. and Wright, D. 2013. Instant 3D design concept generation and visualization by real-time hand gesture recognition. Computers in Industry, 64, 7 (September 2013), 785--797. DOI= http://dx.doi.org/10.1016/j.compind.2013.04.012.

Digital Library

[7]

Leap Motion. Retrieved April 5, 2016, from: https://www.leapmotion.com/.

[8]

Liu, K., Yin, X., Fan, X. and He, Q. 2015. Virtual assembly with physical information: a review. Assembly Automation, 35, 3 (2015), 206--220. DOI= http://dx.doi.org/10.1108/AA-09-2014-074.

Digital Library

[9]

Liu, Z., and Tan, J. 2007. Constrained behavior manipulation for interactive assembly in a virtual environment. The International Journal of Advanced Manufacturing Technology, 32, 7 (April. 2007), 797--810. DOI= http://dx.doi.org/10.1007/s00170-005-0382-5.

[10]

Lok, B., Naik, S., Whitton, M.C. and Brooks, F.P. 2003. Effects of handling real objects and self-avatar fidelity on cognitive task performance and sense of presence in virtual environments. Presence: Teleoper. Virtual Environ. 12, 6 (December 2003), 615--628. DOI=/proxy/http://dx.doi.org/10.1162/105474603322955914.

Digital Library

[11]

Markussen, A., Jakobsen, M. R. and Hornb�k, K. 2014. Vulture: a mid-air word-gesture keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1073--1082. DOI=/proxy/http://dx.doi.org/10.1145/2556288.2556964.

Digital Library

[12]

Microsoft Store. Kinect Sensor for Xbox One. Retrieved April 5, 2016, from: http://www.microsoftstore.com/store/msusa/en_US/pdp/Kinect-for-Xbox-One/productID.307499400.

[13]

Mohler, B.J., Creem-Regehr, S.H., Thompson, W.B. and B�lthoff, H.H. 2010. The effect of viewing a self-avatar on distance judgments in an hmd-based virtual environment. Presence: Teleoper. Virtual Environ. 19, 3 (June 2010), 230--242. DOI= http://dx.doi.org/10.1162/pres.19.3.230.

Digital Library

[14]

Murugappan, S., Liu, H. and Ramani, K. 2013. Shape-It-Up: Hand gesture based creative expression of 3D shapes using intelligent generalized cylinders. Computer Aided Design. 45, 2 (February 2013), 277--287. DOI= http://dx.doi.org/10.1016/j.cad.2012.10.011.

Digital Library

[15]

Nancel, M., Wagner, J., Pietriga, E., Chapuis, O. and Mackay, W. 2011. Mid-air pan-and-zoom on wall-sized displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 177--186. DOI= http://dx.doi.org/10.1145/1978942.1978969.

Digital Library

[16]

Nimble VR. Retrieved April 5, 2016, from: http://nimblevr.com/.

[17]

O'Hagan, R. and Zelinsky, A. 2000. Visual Gesture Interfaces for Virtual Environments. In Proceedings of the First Australasian User Interface Conference (AUIC '00). IEEE Computer Society, Washington, DC, USA, 73.

Digital Library

[18]

Piumsomboon, T., Clark, A., Billinghurst, M. and Cockburn, A. 2013. User-defined gestures for augmented reality. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13). ACM, New York, NY, USA, 955--960. DOI=/proxy/http://dx.doi.org/10.1145/2468356.2468527.

Digital Library

[19]

Raj, M., Creem-Regehr, S.H., Rand, K.M., Stefanucci, J.K. and Thompson, W.B. 2012. Kinect based 3D object manipulation on a desktop display. In Proceedings of the ACM Symposium on Applied Perception (SAP '12). ACM, New York, NY, USA, 99--102. DOI= http://dx.doi.org/10.1145/2338676.2338697.

Digital Library

[20]

Schlattmann, M. and Klein, R. Efficient bimanual symmetric 3D manipulation for bare-Handed interaction. JVRB - Journal of Virtual Reality and Broadcasting, 7, 8 (July 2010).

[21]

Sculpting. Retrieved April 5, 2016, from: https://apps.leapmotion.com/apps/sculpting/windows.

[22]

Segen, J. and Kumar, S. 1998. Gesture VR: vision-based 3D hand interface for spatial interaction. In Proceedings of the sixth ACM international conference on Multimedia (MULTIMEDIA '98). ACM, New York, NY, USA, 455--464. DOI= http://dx.doi.org/10.1145/290747.290822.

Digital Library

[23]

Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., Freedman, D., Kohli, P., Krupka, E., Fitzgibbon, A. and Izadi, S. 2015. Accurate, Robust, and Flexible Real-time Hand Tracking. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3633--3642. DOI= http://dx.doi.org/10.1145/2702123.2702179

Digital Library

[24]

Song, P., Goh, W.B., Hutama, W., Fu, C.W. and Liu, X. 2012. A handle bar metaphor for virtual object manipulation with mid-air interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 1297--1306. DOI= http://dx.doi.org/10.1145/2207676.2208585

Digital Library

[25]

Wang, R., Paris, S. and Popović, J. 2011. 6D hands: markerless hand-tracking for computer aided design. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11). ACM, New York, NY, USA, 549--558. DOI= http://dx.doi.org/10.1145/2047196.204726.

Digital Library

[26]

Wobbrock, J.O., Morris, M.R. and Wilson, A.D. 2009. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 1083--1092. DOI= http://dx.doi.org/10.1145/1518701.1518866.

Digital Library

Cited By

Villarreal-Narvaez SSlu�ters AVanderdonckt JVatavu R(2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
https://dl.acm.org/doi/10.1145/3636458
WEI Z(2023)METAVERSE-BASED ONLINE ENGLISH TEACHING SCHEME IN MULTI-SOURCE AND CROSS-DOMAIN ENVIRONMENTFractals10.1142/S0218348X2340153931:06Online publication date: 13-Jun-2023
https://doi.org/10.1142/S0218348X23401539
Tadeja SSolari Bozzi LSamson KPattinson SBohn� T(2023)Exploring the repair process of a 3D printer using augmented reality-based guidanceComputers and Graphics10.1016/j.cag.2023.10.017117:C(134-144)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.cag.2023.10.017
Show More Cited By

Recommendations

Hitting the Wall: Mid-Air Interaction for Eye-Hand Coordination
VRST '19: Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology

Reaction time training systems are used to improve user performance. Until now, such setups use physical 2D flat surfaces, e.g., a 2D touch screen or buttons mounted on a wall. We designed and investigated a mid-air reaction time training system with an ...
Intelligent sketching interfaces for richer mid-air drawing interactions
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing Systems

Intelligent user interfaces that can understand the content of what we draw in mid-air can enable for exciting novel sketching applications, and for creating more creative designs and engaging content. Currently, existing interaction techniques have ...
Exploring task performance and user's preference of mid-air hand interaction in a 3D docking task experiment
SUI '17: Proceedings of the 5th Symposium on Spatial User Interaction

While the technology for input (Leap Motion, Kinect, etc.) as well as output (VR headsets, large projection walls, etc.) is market ready, only few solutions for natural interaction with such devices exist. With regard to natural 3D interaction, the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CASA '16: Proceedings of the 29th International Conference on Computer Animation and Social Agents

May 2016

200 pages

ISBN:9781450347457

DOI:10.1145/2915926

Conference Chairs:
Nadia Magnenat Thalmann
MIRALab, University of Geneva and NTU Singapore
,
Marina L. Gavrilova
University of Calgary, Canada
,
Program Chairs:
Taku Komura
Edingburgh University, UK
,
Frederic Cordier
University of Haute-Alsace, France
,
Daniel Thalmann
NTU, Singapore & EPFL, Switzerland

Copyright � 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CASA '16

CASA '16: Computer Animation and Social Agents

May 23 - 25, 2016

Geneva, Switzerland

Acceptance Rates

Overall Acceptance Rate 18 of 110 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
380
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Villarreal-Narvaez SSlu�ters AVanderdonckt JVatavu R(2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
https://dl.acm.org/doi/10.1145/3636458
WEI Z(2023)METAVERSE-BASED ONLINE ENGLISH TEACHING SCHEME IN MULTI-SOURCE AND CROSS-DOMAIN ENVIRONMENTFractals10.1142/S0218348X2340153931:06Online publication date: 13-Jun-2023
https://doi.org/10.1142/S0218348X23401539
Tadeja SSolari Bozzi LSamson KPattinson SBohn� T(2023)Exploring the repair process of a 3D printer using augmented reality-based guidanceComputers and Graphics10.1016/j.cag.2023.10.017117:C(134-144)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.cag.2023.10.017
Schwenderling LKleinau AHerbrich WKasireddy HHeinrich FHansen C(2022)Activation modes for gesture-based interaction with a magic lens in AR anatomy visualisationComputer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization10.1080/21681163.2022.215774911:4(1243-1250)Online publication date: 21-Dec-2022
https://doi.org/10.1080/21681163.2022.2157749
Xu WLiang HChen YLi XYu K(2020)Exploring Visual Techniques for Boundary Awareness During Interaction in Augmented Reality Head-Mounted Displays2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR46266.2020.1581268453415(204-211)Online publication date: Mar-2020
https://doi.org/10.1109/VR46266.2020.1581268453415
Xu WLiang HChen YLi XYu K(2020)Exploring Visual Techniques for Boundary Awareness During Interaction in Augmented Reality Head-Mounted Displays2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR46266.2020.00039(204-211)Online publication date: Mar-2020
https://doi.org/10.1109/VR46266.2020.00039
Tadeja SLu YSeshadri PKristensson P(2020)Digital Twin Assessments in Virtual Reality: An Explorational Study with Aeroengines2020 IEEE Aerospace Conference10.1109/AERO47225.2020.9172389(1-13)Online publication date: Mar-2020
https://doi.org/10.1109/AERO47225.2020.9172389
Lupinetti KBonino BGiannini FMonti M(2019)Exploring the Benefits of the Virtual Reality Technologies for Assembly Retrieval ApplicationsAugmented Reality, Virtual Reality, and Computer Graphics10.1007/978-3-030-25965-5_5(43-59)Online publication date: 28-Jul-2019
https://doi.org/10.1007/978-3-030-25965-5_5
Cui JSourin A(2018)Mid-air interaction with optical tracking for 3D modelingComputers & Graphics10.1016/j.cag.2018.04.00474(1-11)Online publication date: Aug-2018
https://doi.org/10.1016/j.cag.2018.04.004
Cui JSourin AGutierrez DHuang H(2017)Interactive shape modeling using leap motion controllerSIGGRAPH Asia 2017 Technical Briefs10.1145/3145749.3149437(1-4)Online publication date: 27-Nov-2017
https://dl.acm.org/doi/10.1145/3145749.3149437
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents