The Locality of Memory Checking
Pages 1820 - 1834
Abstract
Motivated by the extended deployment of authenticated data structures (e.g., Merkle Patricia Tries) for verifying massive amounts of data in blockchain systems, we begin a systematic study of the I/O efficiency of such systems. We first explore the fundamental limitations of memory checking, a previously-proposed abstraction for verifiable storage, in terms of its locality-a complexity measure that we introduce for the first time and is defined as the number of non-contiguous memory regions a checker must query to verifiably answer a read or a write query. Our central result is an Ω(log n/log log n) lower bound for the locality of any memory checker. Then we turn our attention to (dense and sparse) Merkle trees, one of the most celebrated memory checkers, and provide stronger lower bounds for their locality. For example, we show that any dense Merkle tree layout will have average locality at least (1/3)log n. Furthermore, if we allow node duplication, we show that if any write operation has at most polylog complexity, then the read locality cannot be less than log n/log log n. Our lower bounds help us construct two new locality-optimized authenticated data structures (DupTree and PrefixTree) which we implement and evaluate on random operations and real workloads, and which are shown to outperform traditional Merkle trees, especially as the number of leaves increases.
References
[1]
2023. Bitcoin Developer Network. https://bitcoindev.network/.
[2]
Arvind Arasu, Badrish Chandramouli, Johannes Gehrke, Esha Ghosh, Donald Kossmann, Jonathan Protzenko, Ravi Ramamurthy, Tahina Ramananandro, Aseem Rastogi, Srinath Setty, Nikhil Swamy, Alexander van Renen, and Min Xu. 2021. FastVer: Making Data Integrity a Commodity. In Proceedings of the 2021 International Conference on Management of Data (Virtual Event, China) (SIGMOD '21). Association for Computing Machinery, New York, NY, USA, 89--101. https://doi.org/10.1145/3448016.3457312
[3]
Nicola Atzei, Massimo Bartoletti, and Tiziana Cimoli. 2017. A survey of attacks on ethereum smart contracts (sok). In Principles of Security and Trust: 6th International Conference, POST 2017, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2017, Uppsala, Sweden, April 22-29, 2017, Proceedings 6. Springer, 164--186.
[4]
Mihir Bellare, Alexandra Boldyreva, and Adam O'Neill. 2007. Deterministic and efficiently searchable encryption. In Advances in Cryptology-CRYPTO 2007: 27th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 19-23, 2007. Proceedings 27. Springer, 535--552.
[5]
Josh Benaloh and Michael de Mare. 1994. One-Way Accumulators: A Decentralized Alternative to Digital Signatures. In EUROCRYPT '93, Tor Helleseth (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 274--285.
[6]
Manuel Blum, Will Evans, Peter Gemmell, Sampath Kannan, and Moni Naor. 1994. Checking the correctness of memories. Algorithmica, Vol. 12 (1994), 225--244.
[7]
Manuel Blum, William S. Evans, Peter Gemmell, Sampath Kannan, and Moni Naor. 1991. Checking the Correctness of Memories. In 32nd Annual Symposium on Foundations of Computer Science, San Juan, Puerto Rico, 1--4 October 1991. IEEE Computer Society, 90--99.
[8]
Dan Boneh, Benedikt B�nz, and Ben Fisch. 2019. Batching Techniques for Accumulators with Applications to IOPs and Stateless Blockchains. In CRYPTO'19.
[9]
Matteo Campanelli, Dario Fiore, Nicola Greco, Dimitris Kolonelos, and Luca Nizzardo. 2020. Incrementally Aggregatable Vector Commitments and Applications to Verifiable Decentralized Storage. In Advances in Cryptology - ASIACRYPT 2020, Shiho Moriai and Huaxiong Wang (Eds.). Springer International Publishing, Cham, 3--35.
[10]
David Cash, Joseph Jaeger, Stanislaw Jarecki, Charanjit Jutla, Hugo Krawczyk, Marcel-Cua tua lin Rocs u, and Michael Steiner. 2014. Dynamic searchable encryption in very-large databases: Data structures and implementation. Cryptology ePrint Archive (2014).
[11]
David Cash and Stefano Tessaro. 2014. The Locality of Searchable Symmetric Encryption. In EUROCRYPT. Springer, 351--368. https://doi.org/10.1007/978-3-642-55220-5_20
[12]
Dario Catalano and Dario Fiore. 2013. Vector Commitments and Their Applications. In Public-Key Cryptography - PKC 2013 - 16th International Conference on Practice and Theory in Public-Key Cryptography, Nara, Japan, February 26 - March 1, 2013. Proceedings. 55--72.
[13]
Melissa Chase, Apoorvaa Deshpande, Esha Ghosh, and Harjasleen Malvai. 2019. SEEMless: Secure End-to-End Encrypted Messaging with Less Trust. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (London, United Kingdom) (CCS '19). Association for Computing Machinery, New York, NY, USA, 1639--1656. https://doi.org/10.1145/3319535.3363202
[14]
Alexander Chepurnoy, Charalampos Papamanthou, and Yupeng Zhang. 2018. Edrax: A Cryptocurrency with Stateless Transaction Validation. https://eprint.iacr.org/2018/969.
[15]
Ioannis Demertzis, Dimitrios Papadopoulos, and Charalampos Papamanthou. 2018. Searchable encryption with optimal locality: Achieving sublogarithmic read efficiency. In Advances in Cryptology-CRYPTO 2018: 38th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 19-23, 2018, Proceedings, Part I 38. Springer, 371--406.
[16]
Peter J. Denning. 2005. The Locality Principle. Commun. ACM, Vol. 48, 7 (jul 2005), 19--24. https://doi.org/10.1145/1070838.1070856
[17]
Thaddeus Dryja. 2019. Utreexo: A dynamic hash-based accumulator optimized for the Bitcoin UTXO set. https://eprint.iacr.org/2019/611.
[18]
Cynthia Dwork, Moni Naor, Guy N. Rothblum, and Vinod Vaikuntanathan. 2009. How Efficient Can Memory Checking Be?. In Theory of Cryptography, Omer Reingold (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 503--520.
[19]
Ethereum. 2023 a. Etherscan. https://etherscan.io/.
[20]
Ethereum. 2023 b. go-ethereum. https://github.com/ethereum/go-ethereum.
[21]
Ethereum. 2023 a. Optimistic Rollups. https://ethereum.org/en/developers/docs/scaling/optimistic-rollups. Accessed: 2023-05.
[22]
Ethereum. 2023 b. Zero-Knowledge Rollups. https://ethereum.org/en/developers/docs/scaling/zk-rollups/. Accessed: 2023-05.
[23]
Zhenhuan Gao, Yuxuan Hu, and Qinfan Wu. 2021. Jellyfish Merkle Tree. (2021).
[24]
Eran Gilad, Edward Bortnikov, Anastasia Braginsky, Yonatan Gottesman, Eshcar Hillel, Idit Keidar, Nurit Moscovici, and Rana Shahout. 2020. EvenDB: optimizing key-value storage for spatial locality. In Proceedings of the Fifteenth European Conference on Computer Systems. 1--16.
[25]
Google. 2011. LevelDB Benchmarks. http://www.lmdb.tech/bench/microbench/benchmark.html.
[26]
Google. 2018a. LevelDB. https://github.com/google/leveldb. Accessed: 2023-04.
[27]
Google. 2018b. LevelDB - Key Layout. https://github.com/google/leveldb/blob/main/doc/index.md##key-layout. Accessed: 2023-09.
[28]
Google. 2023. Chromium README. https://chromium.googlesource.com/chromium/src//HEAD/third_party/leveldatabase/README.chromium/.
[29]
Sergey Gorbunov, Leonid Reyzin, Hoeteck Wee, and Zhenfei Zhang. 2020. Pointproofs: Aggregating Proofs for Multiple Vector Commitments. In CCS '20: 2020 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, USA, November 9-13, 2020, Jay Ligatti, Xinming Ou, Jonathan Katz, and Giovanni Vigna (Eds.). ACM, 2007--2023.
[30]
David Guichard. 2017. An Introduction to Combinatorics and Graph Theory. Whitman College.
[31]
Aniket Kate, Gregory M. Zaverucha, and Ian Goldberg. 2010. Constant-Size Commitments to Polynomials and Their Applications. In ASIACRYPT'10.
[32]
Polygon Labs. 2022. Introducing Plonky2. https://polygon.technology/blog/introducing-plonky2.
[33]
Chenxing Li, Sidi Mohamed Beillahi, Guang Yang, Ming Wu, Wei Xu, and Fan Long. 2023. LVMT: An Efficient Authenticated Storage for Blockchain. (2023).
[34]
Jiangtao Li, Ninghui Li, and Rui Xue. 2007. Universal Accumulators with Efficient Nonmembership Proofs. In Applied Cryptography and Network Security, Jonathan Katz and Moti Yung (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 253--269.
[35]
Ralph C. Merkle. 1988. A Digital Signature Based on a Conventional Encryption Function. In CRYPTO '87, Carl Pomerance (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 369--378.
[36]
Patrick O'Neil, Edward Cheng, Dieter Gawlick, and Elizabeth O'Neil. 1996. The log-structured merge-tree (LSM-tree). Acta Informatica, Vol. 33 (1996), 351--385.
[37]
Soujanya Ponnapalli, Aashaka Shah, Souvik Banerjee, Dahlia Malkhi, Amy Tai, Vijay Chidambaram, and Michael Wei. 2021. RainBlock: Faster Transaction Processing in Public Blockchains. In USENIX Annual Technical Conference. 333--347.
[38]
Pandian Raju, Soujanya Ponnapalli, Evan Kaminsky, Gilad Oved, Zachary Keener, Vijay Chidambaram, and Ittai Abraham. 2018. mLSM: Making Authenticated Storage Faster in Ethereum. In HotStorage.
[39]
Srinath Setty. 2020. Spartan: Efficient and general-purpose zkSNARKs without trusted setup. In Advances in Cryptology-CRYPTO 2020: 40th Annual International Cryptology Conference, CRYPTO 2020, Santa Barbara, CA, USA, August 17-21, 2020, Proceedings, Part III. Springer, 704--737.
[40]
Dawn Xiaoding Song, David Wagner, and Adrian Perrig. 2000. Practical techniques for searches on encrypted data. In Proceeding 2000 IEEE symposium on security and privacy. S&P 2000. IEEE, 44--55.
[41]
Meta Open Source. 2023. RocksDB. https://rocksdb.org/.
[42]
Shravan Srinivasan, Alexander Chepurnoy, Charalampos Papamanthou, Alin Tomescu, and Yupeng Zhang. 2022. Hyperproofs: Aggregating and Maintaining Proofs in Vector Commitments. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, Boston, MA. https://www.usenix.org/conference/usenixsecurity22/presentation/srinivasan
[43]
Alin Tomescu. 2020. How to Keep a Secret and Share a Public Key (Using Polynomial Commitments). Ph.,D. Dissertation. Massachusetts Institute of Technology, Cambridge, MA, USA.
[44]
Wikipedia contributors. 2023. Binomial theorem - Wikipedia, the free encyclopedia, 2023. https://en.wikipedia.org/wiki/Binomial_theorem. Accessed: 2023-07.
[45]
Gavin Wood et al. 2014. Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper, Vol. 151, 2014 (2014), 1--32.
Index Terms
- The Locality of Memory Checking
Recommendations
Memory Checking Requires Logarithmic Overhead
STOC 2024: Proceedings of the 56th Annual ACM Symposium on Theory of ComputingWe study the complexity of memory checkers with computational security and prove the first general tight lower bound.
Memory checkers, first introduced over 30 years ago by Blum, Evans, Gemmel, Kannan, and Naor (FOCS ’91, Algorithmica ’94), allow a user ...
The complexity of online memory checking
We consider the problem of storing a large file on a remote and unreliable server. To verify that the file has not been corrupted, a user could store a small private (randomized) “fingerprint” on his own computer. This is the setting for the well-...
Minicrypt Primitives with Algebraic Structure and Applications
AbstractAlgebraic structure lies at the heart of much of Cryptomania as we know it. An interesting question is the following: instead of building (Cryptomania) primitives from concrete assumptions, can we build them from simple Minicrypt primitives ...
Comments
Information & Contributors
Information
Published In
November 2023
3722 pages
ISBN:9798400700507
DOI:10.1145/3576915
- General Chairs:
- Weizhi Meng,
- Christian D. Jensen,
- Program Chairs:
- Cas Cremers,
- Engin Kirda
Copyright � 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 November 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
CCS '23
Sponsor:
CCS '23: ACM SIGSAC Conference on Computer and Communications Security
November 26 - 30, 2023
Copenhagen, Denmark
Acceptance Rates
Overall Acceptance Rate 1,261 of 6,999 submissions, 18%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 441Total Downloads
- Downloads (Last 12 months)441
- Downloads (Last 6 weeks)68
Reflects downloads up to 16 Oct 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in