Evaluation-accuracy contract (AA model family, batched paths)
This file states the accuracy contract of the AA evaluation stack in
checkable terms. Enforcement lives in the test suite; every number
below is either asserted by a named test or recorded with the commit
that measured it. No build flag may relax FP semantics anywhere in
this stack: no fast-math, and no compiler-discretionary FP contraction
in the canonical files (cpp/anchor.cpp pins FP_CONTRACT OFF and,
on non-nvcc hosts, -ffp-contract=off).
1. Anchor contractions (tau, J): correctly-rounded-sum semantics
The tau/J GPR contractions are conditioning-limited
(sum_i |krow_i alpha_i| / |result| ~ 1.7e5 for tau, ~1.1e7 for J on
the deployed v2 artifact), so a BLAS dot's value is an accident of its
summation order at the ~1e-8 (tau) / ~1e-9 (J) absolute level, and
anchor errors are adiabatically amplified (~dPhi/dln x ~ 2e3) into
strain phase.
Contract: the anchors are computed by ONE canonical reduction —
Ogita–Rump–Oishi Dot2 (error-free two_prod, Knuth branchless two_sum)
in a FIXED sequential order — implemented twice and required to agree
byte for byte: numpy-EFT (Dekker two_prod,
nrhjsurrogate/physics/anchor.py) and compiled (explicit fma,
cpp/anchor.cpp, host and device execution spaces). For this
conditioning Dot2 is faithfully rounded (within 1 ulp of the exact dot
product), i.e. correctly-rounded-sum semantics: order-independent by
construction, and the fixed order pins the final ulp too.
- Enforcement:
tests/test_anchor_canonical.py- layer (a): all available paths bitwise identical;
- layer (b): golden values (
paper_data/anchor_golden.npz) bitwise from stored inputs; budgeted when kernel rows are recomputed through the local libm (budget DERIVED in-test:4u · Σ_i |krow_i||alpha_im|); - layer (c): conditioning sentinel — the naive contraction's ordering spread × 2e3 must stay below the cross-machine class with ≥ 2x margin (a worse-conditioned retrain fails here first);
- layer (d): FMA/vectorization canary — deliberately variant summations may drift, the canonical paths may not.
One-time evaluation-version shift (2026-08-02, adopting canonical over BLAS-order values): tau max 2.3e-8 abs (3.8e-12 rel), J 4.4e-10 abs; end-to-end native-span strain moved ≤ 1.4e-10 of peak. The canonical values are the defensible ones — the correctly-rounded contraction of the trained coefficients, not an artifact of BLAS blocking.
2. Cross-machine reproducibility limit
Same machine (or stored kernel rows): bitwise. Across machines the
limit is libm — glibc exp is faithfully, not correctly, rounded, so
recomputed kernel rows differ by a few ulp. Propagated through the
conditioning and the 2e3 amplification this bounds cross-machine
DEEP-EXTENSION strain agreement at ~1e-5 relative (J-dominated; tau's
own term is ~1.6e-7). Native-span waveforms are far less sensitive.
This limit is a property of exp + the trained alpha, not of the
reduction; going below it requires a correctly-rounded exp
(crlibm-class), which is out of scope.
3. End-to-end strain parity classes (measured, deployed v2 artifact)
| Comparison | Class | Gate |
|---|---|---|
| batched row vs batch-of-one, same times | ~8e-13 rel | test_aa_batch.py::test_batch_rows_equal_serial (1e-11) |
| fused compiled assembly vs numpy reference | 5.8e-16 of peak | test_fused_assembly_matches_reference (1e-13) |
| kokkos vs numpy backend, native span | ~1e-9..2e-6 of peak (tau-inversion + resampling) | test_vs_numpy_reference (2e-5) |
| batched bilby likelihood vs stock serial | 1.5e-14 rel logL | test_bilby_batch.py (1e-10) |
| float32 opt-in vs float64 | ≤ ~1.9e-4 of peak (~1e-8 mismatch) | test_float32_envelope (5e-4) |
| hybrid (f_low) kokkos vs numpy | 1e-12 (Python pack) / 1e-9 (native pack) | tests/test_kokkos_hybrid.py |
| batched gradients vs scalar chain | bitwise | test_grad_batch_rows_equal_scalar |
Not in this table: the CUDA device path. A NRHJ_BACKEND=cuda build is
verified to COMPILE and link (2026-09-04, GTX 1650 Ti, Turing sm_75, CUDA
13.0), and it carries the Serial and OpenMP host spaces as well. No waveform
has been evaluated on the device and no parity class against the NumPy
reference has been measured, so there is no row for it here and none should
be quoted. The float32 row above is the opt-in low-level evaluator, not the
device path.
Zeros outside a row's span on the shared batch grid are EXACT (the grid-policy contract), asserted in both assembly paths.
4. What is NOT allowed
- fast-math or reassociation flags anywhere in the build;
- FP contraction in
cpp/anchor.cpp(pragma + flag + canary test); - re-introducing order-accidental (BLAS) anchor values on any path —
AAKokkosEvaluator._predict_exactis retained ONLY as the legacy A/B reference for the documented shift.