Enabling Global MPI Process Addressing in MPI Applications
Authors: 
Jean-Baptiste Besnard, 
Sameer Shende, Allen Malony, Julien Jaeger, Marc PeracheAuthors Info & Claims
Jean-Baptiste Besnard, Sameer Shende, Allen Malony, Julien Jaeger, Marc PeracheAuthors Info & ClaimsPages 27 - 36
Abstract
Distributed software using MPI is now facing a complexity barrier. Indeed, given increasing intra-node parallelism, combined with the use of accelerator, programs’ states are becoming more intricate. A given code must cover several cases, generating work for multiple devices. Model mixing generally leads to increasingly large programs and hinders performance portability. In this paper, we pose the question of software composition, trying to split jobs in multiple services. In doing so, we advocate it would be possible to depend on more suitable units while removing the need for extensive runtime stacking (MPI+X+Y). For this purpose, we discuss what MPI shall provide and what is currently available to enable such software composition. After pinpointing (1) process discovery and (2) Remote Procedure Calls (RPCs) as facilitators in such infrastructure, we focus solely on the first aspect. We introduce an overlay-network providing whole-machine inter-job, discovery, and wiring at the level of the MPI runtime. MPI process Unique IDentifiers (UIDs) are then covered as a Unique Resource Locator (URL) leveraged as support for job interaction in MPI, enabling a more horizontal usage of the MPI interface. Eventually, we present performance results for large-scale wiring-up exchanges, demonstrating gains over PMIx in cross-job configurations.
References
[1]
Dong H Ahn, Ned Bass, Albert Chu, Jim Garlick, Mark Grondona, Stephen Herbein, Helgi I Ingólfsson, Joseph Koning, Tapasya Patki, Thomas RW Scogland, 2020. Flux: Overcoming scheduling challenges for exascale workflows. Future Generation Computer Systems 110 (2020), 202–213.
[2]
David E Bernholdt, Swen Boehm, George Bosilca, Manjunath Gorentla Venkata, Ryan E Grant, Thomas Naughton, Howard P Pritchard, Martin Schulz, and Geoffroy R Vallee. 2020. A survey of MPI usage in the US exascale computing project. Concurrency and Computation: Practice and Experience 32, 3(2020), e4851.
[3]
Jean-Baptiste Besnard, Julien Adam, Sameer Shende, Marc Pérache, Patrick Carribault, Julien Jaeger, and Allen D Maloney. 2016. Introducing task-containers as an alternative to runtime-stacking. In Proceedings of the 23rd European MPI Users’ Group Meeting. 51–63.
[4]
Jean-Baptiste Besnard, Allen D Malony, Sameer Shende, Marc Pérache, Patrick Carribault, and Julien Jaeger. 2017. Towards a Better Expressiveness of the Speedup Metric in MPI Context. In 2017 46th International Conference on Parallel Processing Workshops (ICPPW). IEEE, 251–260.
[5]
Ralph H Castain, Joshua Hursey, Aurelien Bouteiller, and David Solt. 2018. Pmix: process management for exascale environments. Parallel Comput. 79(2018), 9–29.
[6]
Bronis R de Supinski, Thomas RW Scogland, Alejandro Duran, Michael Klemm, Sergi Mateo Bellido, Stephen L Olivier, Christian Terboven, and Timothy G Mattson. 2018. The ongoing evolution of openmp. Proc. IEEE 106, 11 (2018), 2004–2019.
[7]
Matthieu Dorier, Matthieu Dreher, Tom Peterka, Justin M Wozniak, Gabriel Antoniu, and Bruno Raffin. 2015. Lessons learned from building in situ coupling frameworks. In Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization. 19–24.
[8]
Michael A Heroux, Rajeev Thakur, Lois McInnes, Jeffrey S Vetter, Xiaoye Sherry Li, James Aherns, Todd Munson, and Kathryn Mohror. 2020. ECP software technology capability assessment report. Technical Report. Oak Ridge National Lab.(ORNL), Oak Ridge, TN (United States).
[9]
Torsten Hoefler, James Dinan, Darius Buntinas, Pavan Balaji, Brian Barrett, Ron Brightwell, William Gropp, Vivek Kale, and Rajeev Thakur. 2013. MPI+ MPI: a new hybrid approach to parallel programming with MPI plus shared memory. Computing 95, 12 (2013), 1121–1136.
[10]
Torsten Hoefler, Andrew Lumsdaine, and Wolfgang Rehm. 2007. Implementation and performance analysis of non-blocking collective operations for MPI. In SC’07: Proceedings of the 2007 ACM/IEEE Conference on Supercomputing. IEEE, 1–10.
[11]
Daniel Holmes, Kathryn Mohror, Ryan E Grant, Anthony Skjellum, Martin Schulz, Wesley Bland, and Jeffrey M Squyres. 2016. Mpi sessions: Leveraging runtime infrastructure to increase scalability of applications at exascale. In Proceedings of the 23rd European MPI Users’ Group Meeting. 121–129.
[12]
Atsushi Hori, Emmanuel Jeannot, George Bosilca, Takahiro Ogura, Balazs Gerofi, Jie Yin, and Yutaka Ishikawa. 2021. An international survey on MPI users. Parallel Comput. 108(2021), 102853.
[13]
Joshua Hursey. 2020. Design considerations for building and running containerized mpi applications. In 2020 2nd International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC). IEEE, 35–44.
[14]
Anuj Kalia, Michael Kaminsky, and David Andersen. 2019. Datacenter {RPCs} can be General and Fast. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19). 1–16.
[15]
A. Kreuzer, E. Suarez, N. Eicker, and Th. Lippert (Eds.). 2021. Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project. Schriften des Forschungszentrums Jülich IAS Series, Vol. 48. Forschungszentrum Jülich GmbH Zentralbibliothek, Verlag, Jülich. 254 pages. https://juser.fz-juelich.de/record/905738
[16]
Eng Keong Lua, Jon Crowcroft, Marcelo Pias, Ravi Sharma, and Steven Lim. 2005. A survey and comparison of peer-to-peer overlay network schemes. IEEE Communications Surveys & Tutorials 7, 2 (2005), 72–93.
[17]
Petar Maymounkov and David Mazieres. 2002. Kademlia: A peer-to-peer information system based on the xor metric. In International Workshop on Peer-to-Peer Systems. Springer, 53–65.
[18]
Marc Pérache, Patrick Carribault, and Hervé Jourdren. 2009. MPC-MPI: An MPI implementation reducing the overall memory consumption. In European Parallel Virtual Machine/Message Passing Interface Users’ Group Meeting. Springer, 94–103.
[19]
Artem Y Polyakov, Boris I Karasev, Joshua Hursey, Joshua Ladd, Mikhail Brinskii, and Elena Shipunova. 2019. A performance analysis and optimization of PMIx-based HPC software stacks. In Proceedings of the 26th European MPI Users’ Group Meeting. 1–10.
[20]
Srinivasan Ramesh, Aurèle Mahéo, Sameer Shende, Allen D Malony, Hari Subramoni, Amit Ruhela, and Dhabaleswar K DK Panda. 2018. MPI performance engineering with the MPI tool interface: the integration of MVAPICH and TAU. Parallel Comput. 77(2018), 19–37.
[21]
Robert B Ross, George Amvrosiadis, Philip Carns, Charles D Cranor, Matthieu Dorier, Kevin Harms, Greg Ganger, Garth Gibson, Samuel K Gutierrez, Robert Latham, 2020. Mochi: Composing data services for high-performance computing environments. Journal of Computer Science and Technology 35, 1 (2020), 121–144.
[22]
Mitsuhisa Sato, Yutaka Ishikawa, Hirofumi Tomita, Yuetsu Kodama, Tetsuya Odajima, Miwako Tsuji, Hisashi Yashiro, Masaki Aoki, Naoyuki Shida, Ikuo Miyoshi, 2020. Co-design for a64fx manycore processor and” fugaku”. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–15.
[23]
Martin Schulz, Marc-André Hermanns, Michael Knobloch, Kathryn Mohror, Nathan T Hjelm, Bengisu Elis, Karlo Kraljic, and Dai Yang. 2021. The MPI Tool Interfaces: Past, Present, and Future—Capabilities and Prospects. In Tools for High Performance Computing 2018/2019. Springer, 55–83.
[24]
Jerome Soumagne, Dries Kimpe, Judicael A Zounmevo, Mohamad Chaarawi, Quincey Koziol, Ahmad Afsahi, and Robert B Ross. 2013. Mercury: Enabling remote procedure call for high-performance computing. In CLUSTER. 1–8.
[25]
Marc-André Vef, Nafiseh Moti, Tim Süß, Tommaso Tocci, Ramon Nou, Alberto Miranda, Toni Cortes, and André Brinkmann. 2018. Gekkofs-a temporary distributed file system for hpc applications. In 2018 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 319–324.
[26]
VÌctor Sande Veiga, Manuel Simon, Abdulrahman Azab, Carlos Fernandez, Giuseppa Muscianisi, Giuseppe Fiameni, and Simone Marocchi. 2019. Evaluation and benchmarking of singularity mpi containers on eu research e-infrastructure. In 2019 IEEE/ACM International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC). IEEE, 1–10.
Index Terms
- Enabling Global MPI Process Addressing in MPI Applications
Recommendations
MT-MPI: multithreaded MPI for many-core environments
ICS '14: Proceedings of the 28th ACM international conference on SupercomputingMany-core architectures, such as the Intel Xeon Phi, provide dozens of cores and hundreds of hardware threads. To utilize such architectures, application programmers are increasingly looking at hybrid programming models, where multiple threads interact ...
Performance Evaluation of MPI Implementations and MPI-Based Parallel ELLPACK Solvers
MPIDC '96: Proceedings of the Second MPI Developers ConferenceAbstract: We are concerned with the parallelization of finite element mesh generation and its decomposition, and the parallel solution of sparse algebraic equations which are obtained from the parallel discretization of second order elliptic partial ...
MPI-StarT: delivering network performance to numerical applications
SC '98: Proceedings of the 1998 ACM/IEEE conference on SupercomputingWe describe an MPI implementation for a cluster of SMPs interconnected by a high-performance interconnect. This work is a collaboration between a numerical applications programmer and a cluster interconnect architect. The collaboration started with the ...
Comments
Information & Contributors
Information
Published In
September 2022
78 pages
ISBN:9781450397995
DOI:10.1145/3555819
Copyright � 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 September 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
EuroMPI/USA'22
Acceptance Rates
Overall Acceptance Rate 66 of 139 submissions, 47%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 82Total Downloads
- Downloads (Last 12 months)21
- Downloads (Last 6 weeks)2
Reflects downloads up to 17 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format