research-article

Towards accurate and practical predictive models of active-vision-based visual search

Authors:

David E. Kieras,

Anthony J. HornofAuthors Info & Claims

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 3875 - 3884

https://doi.org/10.1145/2556288.2557324

Published: 26 April 2014 Publication History

Abstract

Being able to predict the performance of interface designs using models of human cognition and performance is a long-standing goal of HCI research. This paper presents recent advances in cognitive modeling which permit increasingly realistic and accurate predictions for visual human-computer interaction tasks such as icon search by incorporating an "active vision" approach which emphasizes eye movements to visual features based on the availability of features in relationship to the point of gaze. A high fidelity model of a classic visual search task demonstrates the value of incorporating visual acuity functions into models of visual performance. The features captured by the high-fidelity model are then used to formulate a model simple enough for practical use, which is then implemented in an easy-to-use GLEAN modeling tool. Easy-to-use predictive models for complex visual search are thus feasible and should be further developed.

References

[1]

Anstis, S. M. (1974). A chart demonstrating variations in acuity with retinal position. Vision Research, 14(7), 589--592.

[2]

Bellamy, R., John, B., & Kogan, S. (2011). Deploying CogTool: Integrating quantitative usability assessment into real-world software development. In Proceedings of the 2011 International Conference on Software Engineering, 691--700.

Digital Library

[3]

Callander, M., & Zorman, L. (2007). Usability on patrol. In CHI '07 Extended Abstracts, 1709--1714.

Digital Library

[4]

Card, S. K., Moran, T. P., & Newell, A. (1983). The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.

Digital Library

[5]

Cockburn, A., & Gutwin, C. (2009). A predictive model of human performance with scrolling and hierarchical lists. Human-Computer Interaction, 24(3), 273--314.

[6]

Findlay, J. M., & Gilchrist, I. D. (2003). Active Vision: The Psychology of Looking and Seeing. Oxford University Press.

[7]

Fleetwood, M. D., & Byrne, M. D. (2006). Modeling the visual search of displays: A revised ACT-R/PM model of icon search based on eye tracking data. Human-Computer Interaction, 21(2), 153--197.

Digital Library

[8]

Gordon, J., & Abramov, I. (1977). Color vision in the peripheral retina. II. Hue and saturation. J. Opt. Soc. Am., 67(2), 202--207.

[9]

Halverson, T., & Hornof, A. J. (2011). A computational model of "active vision" for visual search in humancomputer interaction. Human-Computer Interaction, 26(4), 285'314.

Digital Library

[10]

Henderson, J. M., & Castelhano, M. S. (2005). Eye movements and visual memory for scenes. In G. Underwood (Ed.), Cognitive Processes in Eye Guidance. (pp. 213--35). Oxford University Press.

[11]

Hornof, A. J., & Kieras, D. E. (1997). Cognitive modeling reveals menu search is both random and systematic. In Proceedings of CHI '97, 107--114.

Digital Library

[12]

John, B., Starr, S., & Utesch, B. (2012). Experiences with collaborative, distributed predictive human performance modeling. In CHI '12 Ext. Abs., 437--452.

Digital Library

[13]

John, B. E., & Kieras, D. E. (1996). Using GOMS for user interface design and evaluation: Which technique' ACM Transactions on Computer-Human Interaction, 3(4), 287--319.

Digital Library

[14]

John, B. E., Prevas, K., Salvucci, D. D., & Koedinger, K. (2004). Predictive human performance modeling made easy. In Proceedings of CHI 2004, 455--462.

Digital Library

[15]

Kieras, D. (2009). The persistent visual store as the locus of 'xation memory in visual search tasks. In R. Cooper & D. Peebles (Eds.), Proceedings of the Ninth International Conference on Cognitive Modeling, UK.

[16]

Kieras, D. (2010). Modeling visual search of displays of many objects: The role of differential acuity and 'xation memory. In D. D. Salvucci & G. Gunzelmann (Eds.) Proceedings of the 10th International Conference on Cognitive Modeling, August 6--8, 2010, Philadelphia.

[17]

Kieras, D. E. (2007). Model-based evaluation. In J. A. Jacko & A. Sears (Eds.), The Human-Computer Interaction Handbook (2nd ed.). (pp. 1191--208). Mahwah, NJ: Lawrence Erlbaum Associates.

[18]

Kieras, D. E., & Marshall, S. P. (2006). Visual availability and 'xation memory in modeling visual search using the EPIC architecture. In Proceedings of the Annual Meeting of the Cognitive Science Society, 423--428.

[19]

Kieras, D. E. (2004). The EPIC Architecture: Principles of Operation. Retrieved from http:// www.eecs.umich.edu/~kieras/docs/EPIC/ EPICPrinOp.pdf

[20]

Kieras, D. E., & Meyer, D. E. (1997). An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-Computer Interaction, 12(4), 391--438.

Digital Library

[21]

Kieras, D. E., Wood, S. D., Abotel, K., & Hornof, A. (1995). GLEAN: A computer-based tool for rapid GOMS model usability evaluation of user interface designs. In Proceedings of UIST '95, ACM, 91--100.

Digital Library

[22]

Maeda, J. (2013, June 12). The future of design is more than making Apple iOS 'at. Wired Magazine. Retrieved from http://www.wired.com/opinion/2013/06/thefuture-of-design-is-more-than-making-apple-ios--at/

[23]

Osga, G., Van Orden K., Campbell, N., Kellmeyer, D., and Lulue D. (2002). Design and Evaluation of War'ghter Task Support Methods in a Multi-Modal Watchstation. Space & Naval Warfare Center, San Diego, Technical Report 1874.

[24]

Peterson, M. S., Kramer, A. F., Wang, R. F., Irwin, D. E., & McCarley, J. S. (2001). Visual search has memory. Psychological Science, 12(4), 287--292.

[25]

Salvucci, D. D., Zuber, M., Beregovaia, E., & Markley, D. (2005). Distract-R: Rapid prototyping and evaluation of in-vehicle interfaces. In Proc. of CHI '05, 581--589.

Digital Library

[26]

Sanders, M. S., & McCormick, E. J. (1987). Human Factors in Engineering and Design (6th ed.). New York, New York: McGraw-Hill.

[27]

Teo, John, & Blackmon. (2012). CogTool-Explorer: A model of goal-directed user exploration that considers information layout. In Proc. of CHI '12, 2479--2488.

Digital Library

[28]

Vera, A. H., John, B. E., Remington, R., Matessa, M., & Freed, M. A. (2005). Automating human-performance modeling at the millisecond level. Human-Computer Interaction, 20, 225'265.

Digital Library

[29]

Williams, L. G. (1966). A Study Of Visual Search Using Eye Movement Recordings. Technical Report, Honeywell Inc., St. Paul Minnesota, Feb. 28, 1966. NTIS AD0629624.

[30]

Williams, L. G. (1967). The effects of target Specification on objects 'xated during visual search. Acta Psychologica, 27, 355--360.

Cited By

Seifi HBhatia AHornbæk K(2024)Charting User Experience in Physical Human–Robot InteractionACM Transactions on Human-Robot Interaction10.1145/365905813:2(1-29)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3659058
Slavuljica ABektas KStrecker JMayer S(2024)NeighboAR: Efficient Object Retrieval using Proximity- and Gaze-based Object Grouping with an AR SystemProceedings of the ACM on Human-Computer Interaction10.1145/36555998:ETRA(1-19)Online publication date: 28-May-2024
https://dl.acm.org/doi/10.1145/3655599
Chandramouli SShi DPutkonen ADe Peuter SZhang SJokinen JHowes AOulasvirta A(2024)A Workflow for Building Computationally Rational Models of Human BehaviorComputational Brain & Behavior10.1007/s42113-024-00208-6Online publication date: 15-Aug-2024
https://doi.org/10.1007/s42113-024-00208-6
Show More Cited By

Index Terms

Towards accurate and practical predictive models of active-vision-based visual search
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Visual Grouping in Menu Interfaces
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Menu interfaces often arrange options into semantic groups. This semantic structure is then usually conveyed to the user by supplementary visual grouping cues. We investigate whether these visual grouping cues actually help users locate items in menus ...
Individualising Graphical Layouts with Predictive Visual Search Models
Special Issue on IUI 2018

In domains where users are exposed to large variations in visuo-spatial features among designs, they often spend excess time searching for common elements (features) on an interface. This article contributes individualised predictive models of visual ...
Subtle cueing for visual search in augmented reality
ISMAR '12: Proceedings of the 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Visual search in augmented reality environments is an important task that can be facilitated through different cueing methods. Current cueing methods rely on explicit cueing, which can potentially reduce visual search performance. In comparison, this ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2014

4206 pages

ISBN:9781450324731

DOI:10.1145/2556288

General Chairs:
Matt Jones
Swansea University, Wales, UK
,
Philippe Palanque
Universit� Paul Sabatier, France
,
Program Chairs:
Albrecht Schmidt
University of Stuttgart, Germany
,
Tovi Grossman
Autodesk Research, Canada

Copyright � 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 April 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '14

Sponsor:

SIGCHI

CHI '14: CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2014

Ontario, Toronto, Canada

Acceptance Rates

CHI '14 Paper Acceptance Rate 465 of 2,043 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
1,597
Total Downloads

Downloads (Last 12 months)122
Downloads (Last 6 weeks)18

Reflects downloads up to 21 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Seifi HBhatia AHornbæk K(2024)Charting User Experience in Physical Human–Robot InteractionACM Transactions on Human-Robot Interaction10.1145/365905813:2(1-29)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3659058
Slavuljica ABektas KStrecker JMayer S(2024)NeighboAR: Efficient Object Retrieval using Proximity- and Gaze-based Object Grouping with an AR SystemProceedings of the ACM on Human-Computer Interaction10.1145/36555998:ETRA(1-19)Online publication date: 28-May-2024
https://dl.acm.org/doi/10.1145/3655599
Chandramouli SShi DPutkonen ADe Peuter SZhang SJokinen JHowes AOulasvirta A(2024)A Workflow for Building Computationally Rational Models of Human BehaviorComputational Brain & Behavior10.1007/s42113-024-00208-6Online publication date: 15-Aug-2024
https://doi.org/10.1007/s42113-024-00208-6
Zhang YTennekes MDe Jong TCurier LCoecke BChen M(2023)Simulation-based Optimization of User Interfaces for Quality-assuring Machine Learning Model PredictionsACM Transactions on Interactive Intelligent Systems10.1145/359455214:1(1-32)Online publication date: 17-May-2023
https://dl.acm.org/doi/10.1145/3594552
Bouwman TBrumby D(2021)Arranging Icons on Small Displays: Do Hexagonal Layouts Improve Search Performance?Proceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132064124564:1(1022-1026)Online publication date: 9-Feb-2021
https://doi.org/10.1177/1071181320641245
Chen XAcharya AOulasvirta AHowes AKitamura YQuigley AIsbister KIgarashi TBj�rn PDrucker S(2021)An Adaptive Model of Gaze-based SelectionProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445177(1-11)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445177
Oulasvirta ADayama NShiripour MJohn MKarrenbauer A(2020)Combinatorial Optimization of Graphical User Interface DesignsProceedings of the IEEE10.1109/JPROC.2020.2969687108:3(434-464)Online publication date: Mar-2020
https://doi.org/10.1109/JPROC.2020.2969687
Jokinen JWang ZSarcar SOulasvirta ARen X(2020)Adaptive feature guidanceInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2019.102376136:COnline publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1016/j.ijhcs.2019.102376
Sun KYu CShi Y(2019)Exploring Low-Occlusion Qwerty Soft Keyboard Using Spatial LandmarksACM Transactions on Computer-Human Interaction10.1145/331814126:4(1-33)Online publication date: 17-Jun-2019
https://dl.acm.org/doi/10.1145/3318141
Simon FRoudaut AIrani PSerrano MBrewster SFitzpatrick GCox AKostakos V(2019)Finding Information on Non-Rectangular InterfacesProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300332(1-8)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300332
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents