GPU Halo Replay: Lossless Twin Simulations for Flexible In Situ Analysis of Stencil-Based Solvers

We introduce GPU Halo Replay, a solver-aware in situ framework for stencil-based applications that creates a lossless simulation “twin” for advanced visualization and analysis. By decoupling analysis from the primary simulation and delegating it to dedicated twin simulations, GPU Halo Replay overcomes the synchronization and fidelity limitations of traditional post hoc and in situ methods. We demonstrate the functionality and feasibility of GPU Halo Replay on the computational fluid dynamics code HARVEY, a massively parallel solver. Compared to existing in situ approaches, it is co-designed with stencil solvers and exposes new analysis patterns. To meet the diverse needs of real-world applications, we developed GPU Halo Replay variants tailored to common usage scenarios. We illustrate the adaptability of these variants across three representative case studies: concurrent simulation replay, spatially-targeted subdomain analysis, and temporally selective domain evaluation. These results illustrate the potential of simulation twins to enhance analysis capabilities without sacrificing performance or fidelity.