research-article

Open access

Fine-Grain Checkpointing with In-Cache-Line Logging

Authors:

Nachshon Cohen,

David T. Aksun,

James R. LarusAuthors Info & Claims

ASPLOS '19: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems

Pages 441 - 454

https://doi.org/10.1145/3297858.3304046

Published: 04 April 2019 Publication History

Abstract

Non-Volatile Memory offers the possibility of implementing high-performance, durable data structures. However, achieving performance comparable to well-designed data structures in non-persistent (transient) memory is difficult, primarily because of the cost of ensuring the order in which memory writes reach NVM.\@ Often, this requires flushing data to NVM and waiting a full memory round-trip time. In this paper, we introduce two new techniques: Fine-Grained Checkpointing, which ensures a consistent, quickly recoverable data structure in NVM after a system failure, and In-Cache-Line Logging, an undo-logging technique that enables recovery of earlier state without requiring cache-line flushes in the normal case. We implemented these techniques in the Masstree data structure, making it persistent and demonstrating the ease of applying them to a highly optimized system and their low (5.9-15.4%) runtime overhead cost.

References

[1]

Hiroyuki Akinaga and Hisashi Shima. Resistive Random Access Memory (ReRAM) Based on Metal Oxides. Proc. IEEE, 98(12):2237--2251, December 2010.

[2]

Joy Arulraj, Justin Levandoski, Umar Farooq Minhas, and Per-AkeLarson. Bztree: a high-performance latch-free range index for non-volatile memory. Proc. VLDB Endow., 11(5):553--565, 2018.

Digital Library

[3]

Kumud Bhandari, Dhruva R. Chakrabarti, and Hans-J. Boehm. Makalu:Fast recoverable allocation of non-volatile memory. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2016, pages 677--694, New York, NY, USA, 2016. ACM.

Digital Library

[4]

Hans-J. Boehm and Sarita V. Adve. Foundations of the c++ concurrency memory model. In Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '08, pages 68--78, New York, NY, USA, 2008. ACM.

Digital Library

[5]

Dhruva R. Chakrabarti, Hans-J. Boehm, and Kumud Bhandari. Atlas:Leveraging locks for non-volatile memory consistency. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA '14, pages 433--452, New York, NY, USA, 2014. ACM.

Digital Library

[6]

Shimin Chen and Qin Jin. Persistent b+-trees in non-volatile main memory. Proc. VLDB Endow., 8(7):786--797, February 2015.

Digital Library

[7]

Joel Coburn, Adrian M. Caulfield, Ameen Akel, Laura M. Grupp, Rajesh K. Gupta, Ranjit Jhala, and Steven Swanson. Nv-heaps: Makingpersistent objects fast and safe with next-generation, non-volatile memories. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pages 105--118, New York, NY, USA, 2011. ACM.

Digital Library

[8]

Nachshon Cohen, David T. Aksun, and James R. Larus. Object-oriented recovery for non-volatile memory. Proc. ACM Program. Lang., 2(OOPSLA): 153: 1--153: 22, October 2018.

Digital Library

[9]

Nachshon Cohen, Michal Friedman, and James R. Larus. Efficient logging in non-volatile memory by exploiting coherency protocols. Proc. ACM Program. Lang., 1(OOPSLA):67:1--67:24, October 2017.

Digital Library

[10]

Krzysztof Czurylo and Andy Rudoff. NVML: NVM Library, 2014.

[11]

Pascal Felber, Christof Fetzer, and Torvald Riegel. Dynamic performance tuning of word-based software transactional memory. In Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP '08, pages 237--246, New York, NY, USA, 2008. ACM.

Digital Library

[12]

M. Hosomi, H. Yamagishi, T. Yamamoto, K. Bessho, Y. Higo, K. Yamane,H. Yamada, M. Shoji, H. Hachino, C. Fukumoto, H. Nagao, and H. Kano. A novel nonvolatile memory with spin torque transfer magnetization switching: spin-ram. In IEEE Int. Devices Meet. 2005. IEDM Tech. Dig., pages 459--462. IEEE, 2005.

[13]

Terry Ching-Hsiang Hsu, Helge Br�gner, Indrajit Roy, Kimberly Keeton, and Patrick Eugster. Nvthreads: Practical persistence for multi-threaded applications. In Proceedings of the Twelfth European Conference on Computer Systems, EuroSys '17, pages 468--482, New York, NY, USA, 2017. ACM.

Digital Library

[14]

Qingda Hu, Jinglei Ren, Anirudh Badam, Jiwu Shu, and Thomas Moscibroda. Log-structured non-volatile main memory. In Proceedings of the 2017 USENIX Conference on Usenix Annual Technical Conference, USENIX ATC '17, pages 703--717. USENIX Association, 2017.

Digital Library

[15]

Intel. Intel and micron produce breakthrough memory technology, 2015.

[16]

Aasheesh Kolli, Steven Pelley, Ali Saidi, Peter M. Chen, and Thomas F. Wenisch. High-performance transactions for persistent memories. In Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS'16, pages 399--411, New York, NY, USA, 2016. ACM.

Digital Library

[17]

Benjamin C. Lee, Engin Ipek, Onur Mutlu, and Doug Burger. Architecting phase change memory as a scalable dram alternative. In Proceedings of the 36th Annual International Symposium on Computer Architecture, ISCA '09, pages 2--13, New York, NY, USA, 2009. ACM.

Digital Library

[18]

Se Kwon Lee, K. Hyun Lim, Hyunsub Song, Beomseok Nam, and Sam H. Noh. Wort: Write optimal radix tree for persistent memory storage systems. In Proceedings of the 15th Usenix Conference on File and Storage Technologies, FAST'17, pages 257--270. USENIX Association, 2017.

Digital Library

[19]

Mengxing Liu, Mingxing Zhang, Kang Chen, Xuehai Qian, YongweiWu, Weimin Zheng, and Jinglei Ren. Dudetm: Building durable trans-actions with decoupling for persistent memory. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS '17, pages 329--343, New York, NY, USA, 2017. ACM.

Digital Library

[20]

Jeremy Manson, William Pugh, and Sarita V. Adve. The java memory model. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposiumon Principles of Programming Languages, POPL '05, pages 378--391, New York, NY, USA, 2005. ACM.

Digital Library

[21]

Yandong Mao, Eddie Kohler, and Robert Tappan Morris. Cache craftiness for fast multicore key-value storage. In Proceedings of the 7th ACM European Conference on Computer Systems, EuroSys '12, pages 183--196, New York, NY, USA, 2012. ACM.

Digital Library

[22]

Amirsaman Memaripour, Anirudh Badam, Amar Phanishayee, Yanqi Zhou, Ramnatthan Alagappan, Karin Strauss, and Steven Swanson. Atomic in-place updates for non-volatile main memories with kamino-tx. In Proceedings of the Twelfth European Conference on Computer Systems, EuroSys '17, pages 499--512, New York, NY, USA, 2017. ACM.

Digital Library

[23]

Faisal Nawab, Joseph Izraelevitz, Terence Kelly, Charles B. Morrey III, Dhruva R. Chakrabarti, and Michael L. Scott. Dal�: A Periodically Persistent Hash Map. In 31st Int. Symp. Distrib. Comput. - DISC 2017, volume 91. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2017.

[24]

Ismail Oukid, Johan Lasperas, Anisoara Nica, Thomas Willhalm, and Wolfgang Lehner. Fptree: A hybrid scm-dram persistent and con-current b-tree for storage class memory. In Proceedings of the 2016 International Conference on Management of Data, SIGMOD '16, pages 371--386, New York, NY, USA, 2016. ACM.

Digital Library

[25]

Moinuddin K. Qureshi, Vijayalakshmi Srinivasan, and Jude A. Rivers. Scalable high performance main memory system using phase-change memory technology. In Proceedings of the 36th Annual International Symposium on Computer Architecture, ISCA '09, pages 24--33, New York, NY, USA, 2009. ACM.

Digital Library

[26]

David Schwalb, Tim Berning, Martin Faust, Markus Dreseler, andHasso Plattner. nvm malloc: Memory allocation for NVRAM. In International Workshop on Accelerating Data Management Systems Using Modern Processor and Storage Architectures - ADMS 2015, Kohala Coast, Hawaii, USA, August 31, 2015., pages 61--72, 2015.

[27]

Stephen Tu, Wenting Zheng, Eddie Kohler, Barbara Liskov, and SamuelMadden. Speedy transactions in multicore in-memory databases. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, SOSP '13, pages 18--32, New York, NY, USA, 2013. ACM.

Digital Library

[28]

Haris Volos, Andres Jaan Tack, and Michael M. Swift. Mnemosyne: Lightweight persistent memory. In Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pages 91--104, New York, NY, USA, 2011. ACM.

Digital Library

[29]

H.-S. Philip Wong, Heng-Yuan Lee, Shimeng Yu, Yu-Sheng Chen, Yi Wu, Pang-Shiu Chen, Byoungil Lee, Frederick T. Chen, and Ming-Jinn Tsai. Metal-Oxide RRAM.Proc. IEEE, 100(6):1951--1970, June 2012.

[30]

Jun Yang, Qingsong Wei, Cheng Chen, Chundong Wang, Khai LeongYong, and Bingsheng He. Nv-tree: Reducing consistency cost for nvm-based single level systems. In Proceedings of the 13th USENIX Conference on File and Storage Technologies, FAST'15, pages 167--181. USENIX Association, 2015.

Digital Library

Cited By

Huang WZhou JWang MZhou YZhang XZhu FLi SWang KWu F(2024)TieredHM: Hotspot-Optimized Hash Indexing for Memory-Semantic SSD-Based Hybrid MemoryIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.335469343:6(1755-1768)Online publication date: Jun-2024
https://doi.org/10.1109/TCAD.2024.3354693
KP AMishra DPanda B(2024)Prosper: Program Stack Persistence in Hybrid Memory Systems2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA57654.2024.00091(1168-1183)Online publication date: 2-Mar-2024
https://doi.org/10.1109/HPCA57654.2024.00091
Soh YSwanson SZhao J(2023)ENTS: Flush-and-Fence-Free Failure Atomic TransactionsProceedings of the International Symposium on Memory Systems10.1145/3631882.3631907(1-16)Online publication date: 2-Oct-2023
https://dl.acm.org/doi/10.1145/3631882.3631907
Show More Cited By

Index Terms

Fine-Grain Checkpointing with In-Cache-Line Logging
1. Hardware
  1. Integrated circuits
    1. Semiconductor memory
      1. Non-volatile memory

Recommendations

DEUCE: Write-Efficient Encryption for Non-Volatile Memories
ASPLOS'15

Phase Change Memory (PCM) is an emerging Non Volatile Memory (NVM) technology that has the potential to provide scalable high-density memory systems. While the non-volatility of PCM is a desirable property in order to save leakage power, it also has the ...
Failure-Atomic Persistent Memory Updates via JUSTDO Logging
ASPLOS'16

Persistent memory invites applications to manipulate persistent data via load and store instructions. Because failures during updates may destroy transient data (e.g., in CPU registers), preserving data integrity in the presence of failures requires ...
Twizzler: A Data-centric OS for Non-volatile Memory
Byte-addressable, non-volatile memory (NVM) presents an opportunity to rethink the entire system stack. We present Twizzler, an operating system redesign for this near-future. Twizzler removes the kernel from the I/O path, provides programs with memory-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASPLOS '19: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems

April 2019

1126 pages

ISBN:9781450362405

DOI:10.1145/3297858

General Chairs:
Iris Bahar
Brown University
,
Maurice Herlihy
Brown University
,
Program Chairs:
Emmett Witchel
University of Texas, Austin
,
Alvin Lebeck
Duke University

Copyright � 2019 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

In-Cooperation

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 April 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ASPLOS '19

Sponsor:

ASPLOS '19: Architectural Support for Programming Languages and Operating Systems

April 13 - 17, 2019

RI, Providence, USA

Acceptance Rates

ASPLOS '19 Paper Acceptance Rate 74 of 351 submissions, 21%;

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
1,543
Total Downloads

Downloads (Last 12 months)243
Downloads (Last 6 weeks)15

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huang WZhou JWang MZhou YZhang XZhu FLi SWang KWu F(2024)TieredHM: Hotspot-Optimized Hash Indexing for Memory-Semantic SSD-Based Hybrid MemoryIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.335469343:6(1755-1768)Online publication date: Jun-2024
https://doi.org/10.1109/TCAD.2024.3354693
KP AMishra DPanda B(2024)Prosper: Program Stack Persistence in Hybrid Memory Systems2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA57654.2024.00091(1168-1183)Online publication date: 2-Mar-2024
https://doi.org/10.1109/HPCA57654.2024.00091
Soh YSwanson SZhao J(2023)ENTS: Flush-and-Fence-Free Failure Atomic TransactionsProceedings of the International Symposium on Memory Systems10.1145/3631882.3631907(1-16)Online publication date: 2-Oct-2023
https://dl.acm.org/doi/10.1145/3631882.3631907
Braun RRamdas AFriedman MAlonso G(2023)PLayer: Expanding Coherence Protocol Stack with a Persistence LayerProceedings of the 1st Workshop on Disruptive Memory Systems10.1145/3609308.3625270(8-15)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3609308.3625270
Miemietz TReusch VRoitzsch MH�rtig H(2023)An NVM Performance Study Towards Whole System Persistence on Server PlatformsProceedings of the 1st Workshop on Disruptive Memory Systems10.1145/3609308.3625269(45-51)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3609308.3625269
Nakata MSato SUgawa T(2023)General-purpose Asynchronous Periodic Checkpointing in Hybrid MemoryProceedings of the 52nd International Conference on Parallel Processing10.1145/3605573.3605648(675-684)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3605573.3605648
Zguem SChen QYeom HHong JLanperne MPark JCerny TShahriar H(2023)NF-Log: Revisiting Log Writes in Relational Database for Efficient Persistent Memory UtilizationProceedings of the 38th ACM/SIGAPP Symposium on Applied Computing10.1145/3555776.3577733(305-312)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3555776.3577733
Zhou JWu JHuang WZhou YWu FShi LZhang XWang KZhu FLi SKloeckner AMoreira J(2022)Tiered HashingProceedings of the International Conference on Parallel Architectures and Compilation Techniques10.1145/3559009.3569652(211-222)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3559009.3569652
Wang QLu YLi JXie MShu J(2022)Nap: Persistent Memory Indexes for NUMA ArchitecturesACM Transactions on Storage10.1145/350792218:1(1-35)Online publication date: 29-Jan-2022
https://dl.acm.org/doi/10.1145/3507922
Khorguani ARopars TDe Palma NBromberg YKermarrec AKozyrakis C(2022)ResPCTProceedings of the Seventeenth European Conference on Computer Systems10.1145/3492321.3519590(525-540)Online publication date: 28-Mar-2022
https://dl.acm.org/doi/10.1145/3492321.3519590
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents