article

Iterative context bounding for systematic testing of multithreaded programs

Authors:

Madanlal Musuvathi,

Shaz QadeerAuthors Info & Claims

ACM SIGPLAN Notices, Volume 42, Issue 6

Pages 446 - 455

https://doi.org/10.1145/1273442.1250785

Published: 10 June 2007 Publication History

Abstract

Multithreaded programs are difficult to get right because of unexpected interaction between concurrently executing threads. Traditional testing methods are inadequate for catching subtle concurrency errors which manifest themselves late in the development cycle and post-deployment. Model checking or systematic exploration of program behavior is a promising alternative to traditional testing methods. However, it is difficult to perform systematic search on large programs as the number of possible program behaviors grows exponentially with the program size. Confronted with this state-explosion problem, traditional model checkers perform iterative depth-bounded search. Although effective for message-passing software, iterative depth-bounding is inadequate for multithreaded software.

This paper proposes iterative context-bounding, a new search algorithm that systematically explores the executions of a multithreaded program in an order that prioritizes executions with fewer context switches. We distinguish between preempting and nonpreempting context switches, and show that bounding the number of preempting context switches to a small number significantly alleviates the state explosion, without limiting the depth of explored executions. We show both theoretically and empirically that context-bounded search is an effective method for exploring the behaviors of multithreaded programs. We have implemented our algorithmin two model checkers and applied it to a number of real-world multithreaded programs. Our implementation uncovered 9 previously unknown bugs in our benchmarks, each of which was exposed by an execution with at most 2 preempting context switches. Our initial experience with the technique is encouraging and demonstrates that iterative context-bounding is a significant improvement over existing techniques for testing multithreaded programs.

References

[1]

Derek Bruening and John Chapin. Systematic testing of multithreaded Java programs. Technical Report LCS-TM-607, MIT/LCS, 2000.

[2]

E.M. Clarke and E.A. Emerson. Synthesis of synchronization skeletons for branching time temporal logic. In Logic of Programs, LNCS 131, pages 52--71. Springer-Verlag, 1981.

Digital Library

[3]

Matthew B. Dwyer, John Hatcliff, Robby, and Venkatesh Prasad Ranganath. Exploiting object excape and locking information in partial-order reductions for concurrent object-oriented programs. Formal Methods in System Design, 25:199--240, 2004.

Digital Library

[4]

Tayfun Elmas, Shaz Qadeer, and Serdar Tasiran. Goldilocks: Efficiently computing the happens-before relation using locksets. In FATES/RV 06: Formal Approaches to Testing and Runtime Verification, volume 4262 of Lecture Notes in Computer Science, pages 193--208. Springer-Verlag, 2006.

Digital Library

[5]

F. Allen Emerson and A. Prasad Sistla. Symmetry and model checking. Formal Methods in System Design, 9(1/2):105--131, August 1996.

Digital Library

[6]

C. Flanagan and S.N. Freund. Atomizer: A dynamic atomicity checker for multithreaded programs. In POPL 04: Principles of Programming Languages, pages 256--267. ACM Press, 2004.

Digital Library

[7]

C. Flanagan and P. Godefroid. Dynamic partial-order reduction for model checking software. In POPL 05: Principles of Programming Languages, pages 110--121. ACM Press, 2005.

Digital Library

[8]

Matteo Frigo, Charles E. Leiserson, and Keith H. Randall. The implementation of the Cilk-5 multithreaded language. In PLDI 98: Programming Language Design and Implementation, pages 212--223. ACM Press, 1998.

Digital Library

[9]

Patrice Godefroid. Partial-Order Methods for the Verification of Concurrent Systems: An Approach to the State-Explosion Problem. LNCS 1032. Springer-Verlag, 1996.

Digital Library

[10]

Patrice Godefroid. Model checking for programming languages using Verisoft. In POPL 97: Principles of Programming Languages, pages 174--186. ACM Press, 1997.

Digital Library

[11]

Alex Groce and Willem Visser. Model checking Java programs using structural heuristics. In ISSTA 02: Software Testing and Analysis, pages 12--21, 2002.

Digital Library

[12]

Radu Iosif. Exploiting heap symmetries in explicit-state model checking of software. In ASE 01: Automated Software Engineering, pages 254--261, 2001.

Digital Library

[13]

C. Norris Ip and David L. Dill. Better verification through symmetry. Formal Methods in System Design, 9(1/2):41--75, 1996.

Digital Library

[14]

Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and Dennis Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. Technical Report MSR-TR-2006-140, Microsoft Research, 2006.

[15]

Daan Leijen. Futures: a concurrency library for C#. Technical Report MSR-TR-2006-162, Microsoft Research, 2006.

[16]

Madanlal Musuvathi, David Park, Andy Chou, Dawson R. Engler, and David L. Dill. CMC: A pragmatic approach to model checking real code. In OSDI 02: Operating Systems Design and Implementation, pages 75--88, 2002.

Digital Library

[17]

Ratan Nalumasu and Ganesh Gopalakrishnan. An efficient partial order reduction algorithm with an alternative proviso implementation. Formal Methods in System Design, 20(3):231--247, May 2002.

Digital Library

[18]

Doron Peled. Partial order reduction: Model-checking using representatives. In MFCS 96: Mathematical Foundations of Computer Science, pages 93--112. Springer-Verlag, 1996.

Digital Library

[19]

S. Qadeer and J. Rehof. Context-bounded model checking of concurrent software. In TACAS 05: Tools and Algorithms for the Construction and Analysis of Systems, volume 3440 of Lecture Notes in Computer Science, pages 93--107. Springer-Verlag, 2005.

Digital Library

[20]

S. Qadeer and D. Wu. KISS: Keep it simple and sequential. In PLDI 04: Programming Language Design and Implementation, pages 14--24. ACM Press, 2004.

Digital Library

[21]

J. Queille and J. Sifakis. Specification and verification of concurrent systems in CESAR. In Fifth International Symposium on Programming, Lecture Notes in Computer Science 137, pages 337--351. Springer-Verlag, 1981.

Digital Library

[22]

Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach (Second Edition). Prentice Hall, 2002.

Digital Library

[23]

Stefan Savage, Michael Burrows, Greg Nelson, Patrick Sobalvarro, and Thomas Anderson. Eraser: a dynamic data race detector for multithreaded programs. ACM Transactions on Computer Systems, 15(4):391--411, 1997.

Digital Library

[24]

Hemanthkumar Sivaraj and Ganesh Gopalakrishnan. Random walk based heuristic algorithms for distributed memory model checking. Electronic Notes in Theoretical Computer Science, 89(1), 2003.

Cited By

Qin FZheng ZSui YGong SShi ZTrivedi K(2024)Cross-project concurrency bug prediction using domain-adversarial neural networkJournal of Systems and Software10.1016/j.jss.2024.112077214(112077)Online publication date: Aug-2024
https://doi.org/10.1016/j.jss.2024.112077
Jeong DJung MLee YLee BShin IKwon YFedorova ANarayanan DDi Luna GQuerzoni L(2023)Diagnosing Kernel Concurrency Failures with AITIAProceedings of the Eighteenth European Conference on Computer Systems10.1145/3552326.3567486(94-110)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3552326.3567486
Ma LZhou DWu HZhou YChang RXiong HWu LRen K(2023)When Top-down Meets Bottom-up: Detecting and Exploiting Use-After-Cleanup Bugs in Linux Kernel2023 IEEE Symposium on Security and Privacy (SP)10.1109/SP46215.2023.10179356(2138-2154)Online publication date: May-2023
https://doi.org/10.1109/SP46215.2023.10179356
Show More Cited By

Recommendations

Iterative context bounding for systematic testing of multithreaded programs
PLDI '07: Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation

Multithreaded programs are difficult to get right because of unexpected interaction between concurrently executing threads. Traditional testing methods are inadequate for catching subtle concurrency errors which manifest themselves late in the ...
Variable and thread bounding for systematic testing of multithreaded programs
ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis

Previous approaches to systematic state-space exploration for testing multi-threaded programs have proposed context-bounding and depth-bounding to be effective ranking algorithms for testing multithreaded programs. This paper proposes two new metrics ...
GAMBIT: effective unit testing for concurrency libraries
PPoPP '10: Proceedings of the 15th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

As concurrent programming becomes prevalent, software providers are investing in concurrency libraries to improve programmer productivity. Concurrency libraries improve productivity by hiding error-prone, low-level synchronization from programmers and ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 42, Issue 6

Proceedings of the 2007 PLDI conference

June 2007

491 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/1273442

Issue’s Table of Contents

PLDI '07: Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation
June 2007
508 pages
ISBN:9781595936332
DOI:10.1145/1250734
General Chair:
Jeanne Ferrante
University of California, San Diego, USA
,
Program Chair:
Kathryn S. McKinley
University of Texas at Austin, USA

Copyright � 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 June 2007

Published in�SIGPLAN�Volume 42, Issue 6

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

463
Total Citations
View Citations
1,239
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)4

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Qin FZheng ZSui YGong SShi ZTrivedi K(2024)Cross-project concurrency bug prediction using domain-adversarial neural networkJournal of Systems and Software10.1016/j.jss.2024.112077214(112077)Online publication date: Aug-2024
https://doi.org/10.1016/j.jss.2024.112077
Jeong DJung MLee YLee BShin IKwon YFedorova ANarayanan DDi Luna GQuerzoni L(2023)Diagnosing Kernel Concurrency Failures with AITIAProceedings of the Eighteenth European Conference on Computer Systems10.1145/3552326.3567486(94-110)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3552326.3567486
Ma LZhou DWu HZhou YChang RXiong HWu LRen K(2023)When Top-down Meets Bottom-up: Detecting and Exploiting Use-After-Cleanup Bugs in Linux Kernel2023 IEEE Symposium on Security and Privacy (SP)10.1109/SP46215.2023.10179356(2138-2154)Online publication date: May-2023
https://doi.org/10.1109/SP46215.2023.10179356
Cao YDong Y(2023)Modeling and Discovering Data Race with Concurrent Code Property Graphs2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C60940.2023.00074(646-653)Online publication date: 22-Oct-2023
https://doi.org/10.1109/QRS-C60940.2023.00074
Blankestijn MLaarman A(2023)Incremental Property Directed ReachabilityFormal Methods and Software Engineering10.1007/978-981-99-7584-6_13(208-227)Online publication date: 9-Nov-2023
https://doi.org/10.1007/978-981-99-7584-6_13
Koval NFedorov ASokolova MTsitelov DAlistarh D(2023)Lincheck: A Practical Framework for�Testing Concurrent Data Structures on�JVMComputer Aided Verification10.1007/978-3-031-37706-8_8(156-169)Online publication date: 17-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-37706-8_8
Deligiannis PSenthilnathan ANayyar FLovett CLal A(2023)Industrial-Strength Controlled Concurrency Testing for Programs with Tools and Algorithms for the Construction and Analysis of Systems10.1007/978-3-031-30820-8_26(433-452)Online publication date: 22-Apr-2023
https://dl.acm.org/doi/10.1007/978-3-031-30820-8_26
Luo NAntonopoulos THarris WPiskac RTromer EWang XYin HStavrou ACremers CShi E(2022)Proving UNSAT in Zero KnowledgeProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security10.1145/3548606.3559373(2203-2217)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3548606.3559373
Wen CHe MWu BXu ZQin SDwyer MDamian DZeller A(2022)Controlled concurrency testing via periodical schedulingProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510178(474-486)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510178
Liu YXu ZFan MHao YChen KChen HCai YYang ZLiu T(2022)ConcSpectre: Be Aware of Forthcoming Malware Hidden in Concurrent ProgramsIEEE Transactions on Reliability10.1109/TR.2022.316269471:2(1174-1188)Online publication date: Jun-2022
https://doi.org/10.1109/TR.2022.3162694
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents