Reduced Models, Surrogates, and Machine Learning

Reduced and learned models accelerate relativistic calculations by exploiting structure in a versioned high-fidelity solution family. This chapter moves from linear compression to empirical-node reconstruction, probabilistic and neural parameter maps, explicit physical constraints, and out-of-distribution failure tests. The governing rule is simple: a surrogate inherits the domain, conventions, numerical error, and omissions of its data, then adds approximation and generalization errors of its own.

Helpful background. Formulations of General Relativity supplies the equations, constraints, gauge content, and symmetries being approximated; Differential Equations for Relativity supplies solution and residual checks; Computational Relativity supplies convergence and reproducible evidence; Probability and Inference for Gravity supplies calibration, held-out validation, and uncertainty metrics.

Enter the surrogate pipeline

Let $h(\lambda)$ be a high-fidelity field or waveform over a declared parameter domain $\Lambda$. The reduced-space projection $P_nh$ is a diagnostic baseline, while an empirical-node surrogate follows the operative path

$$h(\lambda) \longrightarrow \mathcal I_mh(\lambda) \longrightarrow \widehat h(\lambda),$$

where $P_n$ projects onto a reduced basis, $\mathcal I_m$ reconstructs from selected nodes, and the final map predicts the node data at a new $\lambda$. This makes an error budget visible:

$$\begin{aligned} \|h-\widehat h\| &\le\|h-\mathcal I_mh\|+\|\mathcal I_mh-\widehat h\|,\\ \|h-\mathcal I_mh\| &\le(1+\|\mathcal I_m\|)\|h-P_nh\|. \end{aligned}$$

when $\mathcal I_m$ reproduces the reduced space. The projection error therefore diagnoses compressibility, the interpolation norm diagnoses amplification, and the final term contains parameter-model and node-prediction error. High-fidelity discretization and model-form error sit outside these inequalities and must be added separately. Beyond the validated domain, none of the empirical terms is automatically bounded.

The norm is part of the claim. A time-domain norm, detector-weighted mismatch, phase maximum, constraint residual, and worst-case pointwise error answer different questions. Every page therefore reports both its internal training objective and a physically relevant held-out metric.

The six-page route

Read the pages in order when building a fast model from an expensive catalog:

1. Reduced Bases and Proper Orthogonal Decomposition defines the solution manifold, SVD/POD and greedy bases, projection error, training coverage, and leave-region-out check.
2. Empirical Interpolation and Reduced Operators selects physical nodes, measures interpolation conditioning, separates offline from online cost, and tracks projection, node, and fit errors.
3. Gaussian-Process Emulators conditions interpolation on a kernel and training set, tests predictive coverage, and separates GP variance from simulator discrepancy.
4. Neural Surrogates for Relativistic Systems tests median and worst-case error, physical structure, seed stability, and speed at matched accuracy.
5. Physics-Informed Constraints and Symmetries compares soft residual penalties with hard constructions and asks whether small residual actually controls solution error.
6. Out-of-Distribution and Uncertainty Tests increases parameter and model shift, measures false negatives, and requires a surrogate to abstain when its error is not controlled.

The first two pages expose deterministic approximation structure. The next three compare different parameter maps and ways of incorporating physics. The final page tests the warning system that all of them need.

Start with a calculation you can inspect

The first page compresses two vectors in a weighted norm and computes the discarded error exactly. The second reconstructs every affine function from two endpoint values. These examples distinguish choosing a compact space from recovering coefficients cheaply.

The Gaussian-process page then conditions one prediction on one observed value, so its mean and variance can be calculated by hand. The neural page decomposes waveform error into projection and coefficient errors. The physics-informed page constructs a field with a tiny equation residual but order-one error, and the uncertainty page counts false warnings and missed failures explicitly. These small examples explain what the larger waveform benchmarks measure; they require no trained model or specialist software.

One waveform family, tested without leakage

The recurring benchmark is a versioned family of complex compact-binary waveforms with declared modes, parameter coordinates, time or frequency interval, alignment, units, harmonic and polarization conventions, extraction choices, numerical uncertainty, and source checksums. Split by physical parameter point before constructing a basis or fitting preprocessing:

• scattered interpolation points;
• every parameter-domain face and corner;
• a contiguous interior hole;
• an adverse region containing a rapid feature if one exists; and
• points outside the domain, used to test warning or refusal rather than accuracy claims.

Time samples from one waveform never straddle training and test sets. Alignment, normalization, basis construction, empirical nodes, kernel or network selection, stopping, and thresholds see only training and validation data. Final boundary, hole, and OOD cases remain untouched until the finite model is frozen.

For every method, publish median, 95th-percentile, and maximum detector-weighted mismatch; phase and amplitude error; symmetry or continuum residuals; the worst parameter location; training and inference cost; and variation across seeds where optimization is stochastic. Increase catalog coverage, reduced dimension, interpolation nodes or model capacity, optimization tolerance, and arithmetic precision separately. A declining training loss without stable held-out error is not convergence.

The environment record pins data and preprocessing revisions, split manifest, inner product, basis and node order, model architecture or kernel, priors, optimizer, seeds, precision, software and accelerator versions, operating system or container, inference source, and timing method. The clean reconstruction rebuilds the whole chain and recomputes overlaps and physical residuals through an independent analysis path.

Field and collaborators demonstrate the reduced-basis, empirical-node, parameter-fit, and online decomposition in Field et al. 2014, §§ III–V and Figs. 6, 9–11. The chapter treats that architecture as an interpretable baseline, not as a requirement that every surrogate use the same fit.

Choosing a method by the finite claim

POD is appropriate when average compression in a declared measure matters; a greedy basis emphasizes worst training error. Empirical interpolation is valuable when selected operator or waveform values can be evaluated without rebuilding the full state. Gaussian processes provide a conditional covariance and can guide acquisition when their calibration survives held-out tests. Neural maps can be exceptionally fast but need seed, boundary, symmetry, and worst-case evidence. Physics-informed losses can help with sparse data yet can be ill-conditioned and misleading when residual is treated as error.

The method should be chosen after fixing the application tolerance and domain. Compare total resources to reach the same maximum mismatch or physical residual, including data generation, preprocessing, training, warm-up, and inference volume. A slower interpretable model can be preferable when it supplies a reliable bound; a faster model can be preferable when its adverse failures are detectable and the application can abstain.

Published neural-waveform evidence illustrates why averages and maxima must be separated: Khan and Green report strong median accuracy while their largest mismatches cluster near parameter boundaries in Khan and Green 2021, § V.A, Figs. 2–3. Shift benchmarks outside relativity likewise show that uncertainty rankings and calibration can degrade as data move away from training in Ovadia et al. 2019, §§ 4–5 and Figs. 1–5. These are reasons to test boundaries, not reasons to reject learned models wholesale.

What the results establish

The frontier evidence in this chapter was checked through 2026-08-18. Within-domain speed and accuracy do not establish extrapolation. Gaussian-process variance omits shared simulator discrepancy unless that discrepancy is modeled and identified. Neural ensembles measure some training variation, not every physical uncertainty. A small PINN residual does not control error without boundary conditions, norm, conditioning, and independent evaluation. A confident OOD failure is more consequential than a favorable average score.

Principal failure modes are preprocessing leakage, near-duplicate random holdouts, ill-conditioned interpolation, model selection on the test set, seed-sensitive conclusions, broken symmetry or constraints, speed quoted at unequal accuracy, and a warning score that misses the most dangerous case. The validated domain must be narrowed when these cannot be repaired.

Review is required when the source catalog, physical domain, preprocessing, architecture, kernel, constraint, uncertainty method, software or hardware changes; when an independent worst-case or coverage failure appears; or when new evidence materially expands the validated regime. A released surrogate is a versioned scientific object, not a timeless function.

Review the chapter

A neural waveform model has tiny training loss and median test mismatch $2\times10^{-5}$. Its basis was built from the entire catalog before the train/test split, one boundary point has mismatch $2\times10^{-3}$, nominal 95% intervals cover 45% of a contiguous high-spin holdout, and inference is 100 times faster than the full model on a different accelerator. The authors claim a calibrated, physics-informed replacement over and beyond the training domain. Diagnose the claim.

Solution

The basis construction leaked test information, so the test set is not independent; rebuild every preprocessing stage from training data only. The boundary error must be compared with the application’s maximum tolerance, not hidden by the median. Coverage of 45% falsifies the 95% uncertainty claim in the high-spin region, so either recalibrate on untouched data or exclude that region. “Physics-informed” requires independent symmetry, constraint, or equation residuals and does not follow from the label.

Repeat multiple seeds and a leave-region-out test, compare with empirical-interpolation and Gaussian-process baselines at the same data and maximum error, and rerun timing on declared matched hardware with preprocessing, warm-up, batching, and training amortization included. OOD points may test abstention but cannot enlarge the accuracy domain without new high-fidelity validation. The defensible claim is limited to the rebuilt, leakage-free region whose worst error and coverage meet predeclared thresholds.

References

• Field, Scott E., Chad R. Galley, Jan S. Hesthaven, Jason Kaye, and Manuel Tiglio. “Fast Prediction and Evaluation of Gravitational Waveforms Using Surrogate Models.” Physical Review X 4 (2014): 031006. arXiv:1308.3565. DOI.
• Khan, Sebastian, and Rhys Green. “Gravitational-Wave Surrogate Models Powered by Artificial Neural Networks.” Physical Review D 103 (2021): 064015. arXiv:2008.12932. DOI.
• Ovadia, Yaniv, et al. “Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty under Dataset Shift.” In Advances in Neural Information Processing Systems 32. 2019. arXiv:1906.02530.