Install verification: `nrhjsur-verify`

A test suite that ships inside the package, so a user with pip install nrhjsurrogate and the downloaded model files can check that both work, against reference values recorded from the released models. Written 2026-09-02 on feat/eval-release-tests.

What it is

nrhjsurrogate/verify/ is a subpackage: checks.py holds the checks, __init__.py the runner (nrhjsur-verify, python -m nrhjsurrogate.verify), test_installed_model.py thin pytest wrappers (pytest --pyargs nrhjsurrogate.verify), and reference_values.npz the fixture (151 arrays, 116,605 bytes, scripts/generate_verify_reference_values.py). All three entry points call the same functions. The runner has no dependency beyond the package; pytest is optional.

The fixture stores outputs at three parameter points, (1, 0, 0), (3, 0.4, -0.3) and (7, -0.5, 0.6), at RECORDED sample times. The checks re-evaluate at those exact times (the models synthesise directly at any requested time), so nothing depends on a grid. Reference values are regenerated only when a released artifact changes.

What it checks

In order (checks.ALL_CHECKS): package imports; reference values load; which released files are present and where the package searched; each present file's SHA-256 and size against nrhjsurrogate.models.MODELS (a file carrying one released name but another registered digest is reported as that artifact); the action-angle model loads with its merger-ringdown arm; waveform (seven modes, uniform grid, peak near t = 0, recorded span, values); parameter derivatives (modes_grad(with_mr=True): same sample count as the waveform, endpoints within one spacing, values); the model's own error estimate (all seven modes, whole span, amplitude and phase, values); hybridisation (f_low below reach starts earlier, hybridize=False refuses, values in the extended window); dynamics (omega positive and rising, values); remnant (remnant and remnant_physical, values); physical units (start frequency within 5 percent of f_min, strain magnitude at 440 Mpc, values); the adiabatic-angle model loads, its modes match, hull_mask answers; its derivatives (compiled backend only, see limits); the previous-generation files when present; the compiled backend against numpy when built.

Tolerances, with the evidence

checks.TOLERANCE_NATIVE_SPAN = 1e-6 of the per-mode peak for everything in the native span. Measured legitimate differences are far below it: the fast evaluation flavour (safe=False, a proxy for a different summation order) against the shipped reference read 0.0 on the waveform, 1.2e-11 on the error estimate, 1.9e-11 on dynamics, 3.6e-10 on physical units and 1e-11 to 6.8e-10 on derivative rows; the compiled and NumPy evaluators agree to 1.4e-14 on derivatives, a batched and an unbatched contraction to about 1e-10. A wrong-generation artifact moves the waveform by the model's own accuracy class, about 1e-3. TOLERANCE_EXTENSION = 1e-4 for the post-Newtonian extension, ten times the documented cross-machine class of 1e-5 (ACCURACY.md section 2). TOLERANCE_REMNANT = 1e-8 absolute.

The measure is not naive. At (1, 0, 0) the odd-m modes vanish identically and the mass-ratio derivative of every even-m mode vanishes by exchange symmetry, so those arrays are roundoff (7.9e-18 against chi-rows of 1.71); a derivative row is scaled by the mode's largest derivative row and a vanishing array by its largest sibling at that point (checks._relative_to_peak). At the same point the phase uncertainty of the odd-m modes is +inf by construction (amplitude uncertainty over a zero amplitude) and is accepted only there.

Limits

The adiabatic-angle modes_grad evaluates the waveform through the compiled backend even at backend="numpy" (gradients.evaluate_time_grad), so that check is a SKIP on a numpy-only install and no reference values are recorded for it yet (cpp/ was held by another agent and the only compiled builds on the machine predate the current sources).
The suite says nothing about accuracy against numerical relativity. It says the install reproduces the released models' own outputs.
The file-integrity check reads the digests from nrhjsurrogate.models, so the suite follows whichever registry it ships with; it carries no digests of its own.

What it found on its first consumer-route run