Generalize checkpoints to support arbitrary payloads

[timofeymukha]

Dep #2674

This overhauls the chkpt_t type and related functionality.

Old checkpoints should remain valid

Main change and motivation

Previously the type stores a large number of pointers corresponding to various restart fields, like u, v, w, etc. The main issue is that freezes what a checkpoint can contain, unless one goes in and modifies the type. This is a big issue for neko's extensibility, with roots in the legacy custom-format .chkpt format. At the same time the HDF5 checkpoint file, fundamentally, supports a free layout!

In this PR, the chkpt_t now consists of an array of checkpoint_payloads, which are collections of fields, field series, mesh arrays (a 4D array living on an SEM geometry, used by ALE currently), and ordinary arrays. Generic arrays may be replicated or explicitly distributed. All of these are non-owning of course, and just point to data, like in the original type. Schemes and stuff like ALE simply add payloads to the checkpoint.

The hdf5_file, which writes checkpoints, maps each payload to hdf5 group, and the stores the rest as datasets under it.

To load or write a legacy checkpoint file, a compatibility type legacy_checkpoint_view_t is added that stores all the pointers that were previously in the main type checkpoint type. This structure is populated from the new payload layout in a helper routine called chkp_build_legacy_view.

Additional

• Support for interpolation across p is added to the hdf5 checkpoints.
• ALE now supports p-interpolation @arash-mp At least in principle, I am not sure how well an interpolated mass matrix will work and stuff like that.
• New tests.

The code was written mostly with AI, with iterations on the design from my side. I checked everything manually in the end, with the exception of the low-level hdf5 library calls. I am not versed in that, yet it seems those were very sensitive based on the vtkhdf5 stuff. So if @timfelle, @adperezm or @njansson can take an extra look at that during review it would be great.

[Copilot]

Pull request overview

This PR refactors Neko’s checkpointing to be payload-based (arbitrary named collections of fields/series/arrays/mesh-arrays), enabling extensible HDF5 checkpoint layouts while preserving legacy .chkp compatibility via a positional “legacy view”. It also adds restart support across polynomial orders for HDF5 checkpoints (including ALE) and introduces new unit/integration tests plus documentation updates.

Changes:

• Replaces the monolithic chkp_t pointer layout with an array of checkpoint_payload_t registrations, including time-history registration via a dedicated payload.
• Updates HDF5 and legacy .chkp checkpoint I/O to write/read payload hierarchies and map payloads back into the legacy .chkp positional format.
• Adds tests (pFUnit + pytest integration) and updates user documentation/CHANGELOG to reflect new checkpoint formats and capabilities.

Reviewed changes

Copilot reviewed 30 out of 32 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/unit/Makefile.am	Adds checkpoint_payload unit-test subdir and test target.
tests/unit/checkpoint_payload/test_checkpoint_payload.pf	New pFUnit coverage for payload registration and lookup semantics.
tests/unit/checkpoint_payload/Makefile.in	Build rules for the new checkpoint payload unit test.
tests/unit/.gitignore	Ignores the new unit-test executable artifact.
tests/integration/tests/test_scalar_restart/test_scalar_restart.py	New integration test for scalar restart continuity (+ HDF5 p-interp variant).
tests/integration/tests/test_scalar_restart/scalar_restart.case	New case file used by scalar restart integration tests.
tests/integration/tests/test_scalar_restart/reference_part1_cpu_sp.csv	Reference convergence data for scalar restart (part 1, CPU SP).
tests/integration/tests/test_scalar_restart/reference_part1_cpu_dp.csv	Reference convergence data for scalar restart (part 1, CPU DP).
tests/integration/tests/test_scalar_restart/reference_part2_cpu_sp.csv	Reference convergence data for scalar restart (part 2, CPU SP).
tests/integration/tests/test_scalar_restart/reference_part2_cpu_dp.csv	Reference convergence data for scalar restart (part 2, CPU DP).
tests/integration/tests/test_ale_restart/test_ale_restart.py	New integration test for ALE HDF5 restart across polynomial orders.
tests/integration/.gitignore	Keeps scalar restart reference CSVs tracked despite global CSV ignore.
src/simulation_components/user_stats.f90	Minor import alignment/formatting adjustment.
src/scalar/scalars.f90	Moves scalar IC handling into scalars_t and removes legacy scalar checkpoint registration logic.
src/scalar/scalar_scheme.f90	Adds payload registration + scalar IC setter; changes params handling to an owned copy.
src/scalar/scalar_scheme_fctry.f90	Ensures scalar schemes register checkpoint payloads after init.
src/scalar/scalar_pnpn.f90	Registers Pn/Pn-specific checkpoint fields and switches time-history access to the new API.
src/Makefile.am	Adds common/checkpoint_payload.f90 to build sources.
src/io/hdf5_file.F90	Implements payload read/write for checkpoints and adds p-interpolation read paths.
src/io/chkp_file.f90	Adds legacy_checkpoint_view_t and maps payloads into legacy .chkp layout.
src/fluid/fluid_scheme_incompressible.f90	Adds validation checks for consistent velocity spaces / mesh compatibility.
src/fluid/fluid_pnpn.f90	Registers fluid checkpoint payload and switches time-history access to the new API.
src/common/checkpoint.f90	Overhauls chkp_t to manage payload registrations and time-history payload.
src/common/checkpoint_payload.f90	New payload container type supporting fields/series/arrays/mesh-arrays.
src/case.f90	Registers time-history via checkpoint API and delegates scalar IC setup to scalars_t.
src/ale/ale_manager.f90	Registers ALE state via payloads and removes legacy-format restriction.
src/.depends	Updates build dependencies for the new payload module and modified compilation units.
doc/pages/user-guide/io.md	Documents .chkp vs HDF5 checkpoints and new capabilities/limits.
doc/pages/user-guide/case-file.md	Updates restart_file suffix rules and ALE restart documentation.
configure.ac	Adds the new unit-test Makefile to autotools generation.
CHANGELOG.md	Adds an entry describing the new payload-based checkpointing and HDF5 enhancements.
.gitignore	Ignores example-generated .h5 files.

Comments suppressed due to low confidence (2)

src/io/hdf5_file.F90:752

• The interpolation path reads the checkpoint data into a 1D buffer (checkpoint_data(:)) and then calls interpolator_t%map_host, whose interface requires rank-4 arrays shaped by the source/target spaces. As written, this is a rank mismatch that should not compile, and space_interp%init(fld%Xh, source_Xh) appears to initialise the interpolator with source/target spaces reversed. Use a local block to remap the flat buffer to a 4D pointer and initialise the interpolator as (source_Xh, fld%Xh) before mapping into fld%x.

       ! Interpolation across spaces required!
       dcount(1) = int(fld%msh%nelv, hsize_t) * &
            int(source_Xh%lxyz, hsize_t)
       doffset(1) = int(fld%msh%offset_el, hsize_t) * &
            int(source_Xh%lxyz, hsize_t)

src/io/hdf5_file.F90:916

• In the mesh-array interpolation branch, stored_data(:) / array%x(:) are rank-1 arrays, but interpolator_t%map_host requires rank-4 arrays. This should not compile (rank mismatch), and space_interp%init(array%Xh, stored_Xh) appears to reverse the source/target spaces. Remap both the stored buffer and array%x to rank-4 pointers in a local block, then initialise the interpolator as (stored_Xh, array%Xh) and map into the remapped destination view.

    else
       allocate(stored_data(int(dcount(1))))
       call h5dread_f(dset_id, h5_neko_real, stored_data, dcount, ierr, &
            file_space_id = filespace, mem_space_id = memspace, &
            xfer_prp = plist_id)

[arash-mp]

Very nice @timofeymukha.
regarding ALE restart with a different poly order, it was already possible with .chkp, but with the (accuracy) cost of setting Blag and Blaglag = B at the restart. Considering that changing poly order will anyway introduce some error and is not considered exact restart. I haven't tested interpolating mass matrix either (not sure even if that's a good idea)
I haven't checked the PR yet, but if it now interpolates the Blag and Blaglag if poly order is changed, I think it's better to leave that as an option, if possible.
also a general question, does it affect both chkp and h5 formats, or works only with h5?

[njansson]

Great stuff 🍻 🐈‍⬛ 🐈