TY - GEN
T1 - Recovery of Distributed Iterative Solvers for Linear Systems Using Non-Volatile RAM
AU - Fridman, Yehonatan
AU - Snir, Yaniv
AU - Levin, Harel
AU - Hendler, Danny
AU - Attiya, Hagit
AU - Oren, Gal
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022/1/1
Y1 - 2022/1/1
N2 - HPC systems are a critical resource for scientific research and advanced industries. The increased demand for computational power and memory ushers in the exascale era, in which supercomputers are designed to provide enormous computing power to meet these needs. These complex supercomputers consist of numerous compute nodes and are consequently expected to experience frequent faults and crashes. Mathematical solvers, in particular, iterative linear solvers are key building block in numerous large-scale scientific applications. Consequently, supporting the recovery of distributed solvers is necessary for scaling scientific applications to exascale platforms. Previous recovery methods for iterative solvers are based on Checkpoint-Restart (CR), which incurs high fault tolerance overhead, or intrinsic fault tolerance, which require extra computation time to converge after failures. Exact state reconstruction (ESR) was proposed as an alternative mechanism to alleviate the impact of frequent failures on long-term computations. ESR has been shown to provide exact reconstruction of the computation state while avoiding the need for costly checkpointing. However, ESR currently relies on volatile memory for fault tolerance, and must therefore maintain redundancies in the RAM of multiple nodes. This not only incurs high memory overhead but also prevents ESR from being fully resilient, that is, resilient against a full system crash. Recent supercomputer designs feature emerging non-volatile RAM (NVRAM) technology, for example, the exascale Aurora that is planned to consist of Intel Optane™ DCPMM. This paper investigates how NVRAM can be utilized to devise an enhanced ESR-based recovery mechanism that is more efficient and provides full resilience. Our mechanism, called in-NVRAM ESR, provides full resiliency while significantly reducing both the memory footprint and the time overhead in comparison with the original ESR design (in-RAM ESR). In-NVRAM ESR is based on a novel MPI One-Sided Communication (OSC) over RDMA implementation, which was optimized and applied for using NVRAM to store recovery data for iterative linear solvers.
AB - HPC systems are a critical resource for scientific research and advanced industries. The increased demand for computational power and memory ushers in the exascale era, in which supercomputers are designed to provide enormous computing power to meet these needs. These complex supercomputers consist of numerous compute nodes and are consequently expected to experience frequent faults and crashes. Mathematical solvers, in particular, iterative linear solvers are key building block in numerous large-scale scientific applications. Consequently, supporting the recovery of distributed solvers is necessary for scaling scientific applications to exascale platforms. Previous recovery methods for iterative solvers are based on Checkpoint-Restart (CR), which incurs high fault tolerance overhead, or intrinsic fault tolerance, which require extra computation time to converge after failures. Exact state reconstruction (ESR) was proposed as an alternative mechanism to alleviate the impact of frequent failures on long-term computations. ESR has been shown to provide exact reconstruction of the computation state while avoiding the need for costly checkpointing. However, ESR currently relies on volatile memory for fault tolerance, and must therefore maintain redundancies in the RAM of multiple nodes. This not only incurs high memory overhead but also prevents ESR from being fully resilient, that is, resilient against a full system crash. Recent supercomputer designs feature emerging non-volatile RAM (NVRAM) technology, for example, the exascale Aurora that is planned to consist of Intel Optane™ DCPMM. This paper investigates how NVRAM can be utilized to devise an enhanced ESR-based recovery mechanism that is more efficient and provides full resilience. Our mechanism, called in-NVRAM ESR, provides full resiliency while significantly reducing both the memory footprint and the time overhead in comparison with the original ESR design (in-RAM ESR). In-NVRAM ESR is based on a novel MPI One-Sided Communication (OSC) over RDMA implementation, which was optimized and applied for using NVRAM to store recovery data for iterative linear solvers.
KW - ESR
KW - Exascale
KW - HPC
KW - Intel Optane DCPMM
KW - Iterative Solvers
KW - MPI OSC
KW - NVRAM
KW - PCG
KW - RDMA
KW - Recovery
UR - https://www.scopus.com/pages/publications/85147947653
U2 - 10.1109/FTXS56515.2022.00007
DO - 10.1109/FTXS56515.2022.00007
M3 - Conference contribution
AN - SCOPUS:85147947653
T3 - Proceedings of FTXS 2022: Workshop on Fault Tolerance for HPC at eXtreme Scale, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis
SP - 11
EP - 23
BT - Proceedings of FTXS 2022
PB - Institute of Electrical and Electronics Engineers
T2 - 12th IEEE/ACM Workshop on Fault Tolerance for HPC at eXtreme Scale, FTXS 2022
Y2 - 13 November 2022 through 18 November 2022
ER -