research-article

Data flow metrics in program comprehension tasks

Authors:

Martin Konopka,

Adam Talian,

Jozef Tvarozek,

Pavol NavratAuthors Info & Claims

EMIP '18: Proceedings of the Workshop on Eye Movements in Programming

Article No.: 2, Pages 1 - 6

https://doi.org/10.1145/3216723.3216728

Published: 15 June 2018 Publication History

Get Access

Abstract

Existing research in program comprehension has paid less attention to the coverage of programming concepts that were contained within the source codes used for studies. In this paper, we examine source codes covering four introductory programming concepts: branching, loops and arrays, sorting, and tail recursion. The diverse types of code fragments give rise to eye movement patterns more structured according to the control flow and data flow of the program. To facilitate analysis of this class of program comprehension strategies, we propose data flow-based metrics and describe automatic computation of the metrics. In evaluation of the proposal, we conducted a pilot study with novice and intermediate programmers. In the study with recordings from 26 programmers we compute basic fixation and saccade metrics along with a data flow-based metric.

References

[1]

Teresa Busjahn, Roman Bednarik, Andrew Begel, Martha Crosby, James H. Paterson, Carsten Schulte, Bonita Sharif, and Sascha Tamm. 2015. Eye Movements in Code Reading: Relaxing the Linear Order. In 2015 IEEE 23rd International Conference on Program Comprehension. IEEE, 255--265.

Crossref

Google Scholar

[2]

Teresa Busjahn, Carsten Schulte, Bonita Sharif, Simon, Andrew Begel, Michael Hansen, Roman Bednarik, Paul Orlov, Petri Ihantola, Galina Shchekotova, and Maria Antropova. 2014. Eye tracking in computing education. In Proceedings of the 10th annual conference on International computing education research (ICER '14). ACM, New York, NY, USA, 3--10.

Crossref

Google Scholar

[3]

Ahmad Jbara and Dror G. Feitelson. 2015. How Programmers Read Regular Code: A Controlled Experiment Using Eye Tracking. In Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension (ICPC '15). IEEE Press, Piscataway, NJ, USA, 244--254.

Crossref

Google Scholar

[4]

Martin Konopka, Robert Moro, Peter Demcak, Patrik Hlavac, Jozef Tvarozek, Jakub Simko, Eduard Kuric, Pavol Navrat, and Maria Bielikova. 2017. Infrastructure and Methodology for Group Studies in Multiple Eye Trackers Laboratory. Journal of Eye Movement Research 10, 6 (2017), p. 292. Special Issue ECEM 2017, Abstract.

Google Scholar

[5]

Pavol Navrat and Jozef Tvarozek. 2014. Online programming exercises for summative assessment in university courses. In Proceedings of the 15th International Conference on Systems and Technologies (CompSysTech '14). ACM, New York, NY, USA, 341--348.

Crossref

Google Scholar

[6]

Unaizah Obaidellah, Mohammmed Al Haek, and Peter C.-H. Cheng. 2018. A Survey on the Usage of Eye-Tracking in Computer Programming. Comput. Surveys 51, 1 (2018), 5:1--5:58.

Digital Library

Google Scholar

[7]

Zohrer Sharafi, Timothy Shaffer, Bonita Sharif, and Yann-Ga�l Gu�h�neuc. 2015. Eye-tracking Metrics in Software Engineering. In 2015 Asia-Pacific Software Engineering Conference (APSEC). IEEE, 1530--1362.

Google Scholar

[8]

Bonita Sharif, Michael Falcone, and Jonathan I. Maletic. 2012. An eye-tracking study on the role of scan time in finding source code defects. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '12). ACM, New York, NY, USA, 381--384.

Digital Library

Google Scholar

[9]

Bonita Sharif and Jonatahn I. Maletic. 2010. An Eye Tracking Study on camelCase and under_score Identifier Styles. In 2010 IEEE 18th International Conference on Program Comprehension. 196--205.

Crossref

Google Scholar

[10]

Jozef Tvarozek, Martin Konopka, Jakub Hucko, Pavol Navrat, and Maria Bielikova. 2017. Robust Recording of Program Comprehension Studies with Eye Tracking for Re-peatable Analysis and Replay. Journal of Eye Movement Research 10, 6 (2017), p. 293. Special Issue ECEM 2017, Abstract.

Google Scholar

[11]

Hidetake Uwano, Masahide Nakamura, Akito Monden, and Ken ichi Matsumoto. 2007. Exploiting Eye Movements for Evaluating Reviewer's Performance in Software Review. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. E90-A, 10 (Oct. 2007), 2290--2300.

Crossref

Google Scholar

[12]

Leelakrishna Yenigalla, Vinayak Sinha, Bonita Sharif, and Martha Crosby. 2016. How Novices Read Source Code in Introductory Courses on Programming: An Eye-Tracking Experiment. In Proceedings of the 10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - Volume 9744, Part II. Springer-Verlag New York, Inc., New York, NY, USA, 120--131.

Crossref

Google Scholar

Cited By

View all

Grabinger LHauser FWolff CMottok J(2024)On Eye Tracking in Software EngineeringSN Computer Science10.1007/s42979-024-03045-35:6Online publication date: 26-Jul-2024
https://dl.acm.org/doi/10.1007/s42979-024-03045-3
Wyrich MBogner JWagner S(2023)40 Years of Designing Code Comprehension Experiments: A Systematic Mapping StudyACM Computing Surveys10.1145/362652256:4(1-42)Online publication date: 9-Nov-2023
https://dl.acm.org/doi/10.1145/3626522
Harada HNakayama M(2021)Estimation of reading ability of program codes using features of eye movementsACM Symposium on Eye Tracking Research and Applications10.1145/3448018.3457421(1-5)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448018.3457421
Show More Cited By

Index Terms

Data flow metrics in program comprehension tasks
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education

Recommendations

What Drives the Reading Order of Programmers?: An Eye Tracking Study
ICPC '20: Proceedings of the 28th International Conference on Program Comprehension

Background: The way how programmers comprehend source code depends on several factors, including the source code itself and the programmer. Recent studies showed that novice programmers tend to read source code more like natural language text, whereas ...
Beyond gaze: preliminary analysis of pupil dilation and blink rates in an fMRI study of program comprehension
EMIP '18: Proceedings of the Workshop on Eye Movements in Programming

Researchers have been employing psycho-physiological measures to better understand program comprehension, for example simultaneous fMRI and eye tracking to validate top-down comprehension models. In this paper, we argue that there is additional value in ...
Eye tracking analysis of computer program comprehension in programmers with dyslexia

This paper investigates the impact of dyslexia on the reading and comprehension of computer program code. Drawing upon work from the fields of program comprehension, eye tracking, dyslexia, models of reading and dyslexia gaze behaviour, a set of ...

Comments

Information & Contributors

Information

Published In

EMIP '18: Proceedings of the Workshop on Eye Movements in Programming

June 2018

35 pages

ISBN:9781450357920

DOI:10.1145/3216723

General Chairs:
Roman Bednarik
University of Eastern Finland
,
Carsten Schulte
Universit�t Paderborn, Germany

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Agent�ra na Podporu V�skumu a V�voja
Vedeck� Grantov� Agent�ra M`VVa` SR a SAV
Ministerstvo akolstva, vedy, v?skumu a aportu Slovenskej republiky
Kult�rna a Edukacn� Grantov� Agent�ra M`VVa` SR

Conference

ETRA '18

Sponsor:

ETRA '18: 2018 Symposium on Eye Tracking Research and Applications

June 15, 2018

Warsaw, Poland

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
216
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Grabinger LHauser FWolff CMottok J(2024)On Eye Tracking in Software EngineeringSN Computer Science10.1007/s42979-024-03045-35:6Online publication date: 26-Jul-2024
https://dl.acm.org/doi/10.1007/s42979-024-03045-3
Wyrich MBogner JWagner S(2023)40 Years of Designing Code Comprehension Experiments: A Systematic Mapping StudyACM Computing Surveys10.1145/362652256:4(1-42)Online publication date: 9-Nov-2023
https://dl.acm.org/doi/10.1145/3626522
Harada HNakayama M(2021)Estimation of reading ability of program codes using features of eye movementsACM Symposium on Eye Tracking Research and Applications10.1145/3448018.3457421(1-5)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448018.3457421
Nakayama MHarada H(2020)Eye Movement Features in response to Comprehension Performance during the Reading of ProgramsACM Symposium on Eye Tracking Research and Applications10.1145/3379156.3391981(1-5)Online publication date: 2-Jun-2020
https://dl.acm.org/doi/10.1145/3379156.3391981

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

What Drives the Reading Order of Programmers?: An Eye Tracking Study

Beyond gaze: preliminary analysis of pupil dilation and blink rates in an fMRI study of program comprehension

Eye tracking analysis of computer program comprehension in programmers with dyslexia

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Upcoming Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

What Drives the Reading Order of Programmers?: An Eye Tracking Study

Beyond gaze: preliminary analysis of pupil dilation and blink rates in an fMRI study of program comprehension

Eye tracking analysis of computer program comprehension in programmers with dyslexia

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations