Residual and Consistency Tests

Residual tests ask whether calibrated data contain coherent structure left unexplained by a fitted GR signal and noise model. Consistency tests ask whether different data subsets infer compatible physical parameters. Both are deliberately broad diagnostics: they can reveal model failure without deciding whether the cause is gravity, source physics, calibration, selection, or noise.

Required background. Weak Metric Perturbations and Gauge Freedom supplies weak-field gravity and waveform perturbations. Probability and Inference for Gravity supplies likelihoods, posterior prediction, covariance, and calibrated test statistics.

Helpful background. Relativistic Two-Body Dynamics supplies the source parameters and inspiral–merger relations whose consistency is tested.

Residuals live inside a noise model

For detector data $d$ and signal $h(\theta)$, define

$$r=d-h(\theta).$$

This subtraction is not yet a test. In stationary Gaussian noise with covariance $C$, the whitened residual $r_w=C^{-1/2}r$ has a declared reference distribution. In real data, the power spectral density, calibration transfer function, glitches, gating or inpainting, data segment, and window all enter the residual. Estimating $C$ from the same segment can also change the null distribution.

A posterior-predictive residual test draws $\theta$ and nuisance quantities from their posterior, generates replicated data through the calibrated likelihood, and compares a predeclared statistic $T(r)$ with the observed value. Useful statistics include coherent network energy, time–frequency morphology, autocorrelation, and projection onto waveform-derivative directions. Searching many pixels, wavelets, start times, or frequency bands requires the corresponding trials to be included in calibration.

To see what fitting changes, take a fixed, correctly whitened linear model $d=Xa+n$, with $n\sim\mathcal N(0,I)$ and $X$ of column rank $p$. Least squares gives the projection

$$P=X(X^TX)^{-1}X^T,\qquad r=(I-P)d=(I-P)n.$$

Since $P^2=P=P^T$, the residual covariance is $I-P$, not $I$. Only $N-p$ independent noise directions remain, and $r^Tr\sim\chi^2_{N-p}$. A missing deterministic component $b$ instead produces a noncentral chi-square law with noncentrality $b^T(I-P)b$. A component lying entirely in the fitted space leaves no residual at all: it biases fitted parameters. These exact statements assume fixed $X$ and known covariance. Posterior-predictive tail areas reuse fitted data and are not generally uniform; do not give them the repeated-sampling interpretation of the exact chi-square tail below without a separate calibration.

Unmodeled coherent power is evidence against the composite fitted model. It is not automatically evidence against GR. A missing higher harmonic, eccentricity, precession, scattered-light transient, calibration line, or inaccurate merger model can produce a structured residual.

Split-data consistency

Let subsets $A$ and $B$ produce estimates of a common parameter vector. In a Gaussian approximation,

$$\Delta\theta=\hat\theta_A-\hat\theta_B, \qquad Q=\Delta\theta^T C_\Delta^{-1}\Delta\theta,$$

with

$$C_\Delta=C_A+C_B-C_{AB}-C_{AB}^T.$$

Here $C_A$, $C_B$, and $C_{AB}$ must describe the sampling covariance of these estimates under the null for $Q$ to have a frequentist chi-square interpretation. In the regular unbiased Gaussian case, its degrees of freedom are the number of identifiable difference directions; singular directions require restriction to their support or a pseudoinverse, not an ordinary inverse. Shared calibration, noise estimates, and overlapping data can create cross-covariance. Using the same fixed prior can also shrink both estimates without making that prior an additional random measurement. Posterior covariance is not automatically sampling covariance. An inspiral–post-inspiral comparison, detector split, observing-run split, or frequency split is meaningful only after the split rule and common parameter map are fixed. Choosing the split after examining the discrepancy invalidates its nominal reference distribution.

The current GWTC-5 tests manuscript deliberately omits the traditional inspiral–merger–ringdown consistency test because studies found spurious biases under its then-used implementation; dispersive propagation is also deferred. This is an instructive negative result: a familiar statistic is not retained merely because it once had a standard label LVK 2026, P2500781-v17, Table 1 and §§2–3.

Dataset and analysis assumptions

For a real gravitational-wave residual analysis at the 2026-08-19 evidence cutoff, pin the exact event catalog, strain and parameter-estimation release, calibration version, sample rate, and file hashes. The current public catalog baseline is GWTC-5 stable parameter-estimation release 9, DOI 10.7935/bk00-6a89; its 16 kHz products are source-matched, while 4 kHz products are SciPy-decimated with a documented filter error, and data below 10 Hz are not calibrated GWOSC 2026, GWTC-5.0 data notes.

State all event, detector, data-quality, signal-to-noise, time, and frequency selections. If events enter because a residual statistic was interesting, the selection is outcome-dependent and a population conclusion is biased. The likelihood must normalize the noise model, propagate calibration covariance, declare waveform and glitch priors, and include sampler-convergence diagnostics. Reproduction requires released inputs, software commit, dependency lock, configuration, random seeds, and machine-readable residual statistics.

No real-data numerical result is reproduced on this method page. Instead, the following controlled benchmark specifies the minimum behavior an implementation must demonstrate before it is applied to a catalog.

A reproducible null benchmark

Use Python 3.12.5, NumPy 2.1.0, and SciPy 1.14.1 with random seed 17017. Generate $N$ unit-variance Gaussian samples plus a known linear template $a x$, fit $a$, and evaluate $Q=r^Tr$ against $\chi^2_{N-1}$. Repeat for a null injection and for a fault containing an omitted orthogonal component. A compact reference calculation is:

import numpy as np
from scipy.stats import chi2

rng = np.random.default_rng(17017)

def one_trial(n=4096, fault=0.0):
    phase = 2 * np.pi * np.arange(n) / n
    x = np.cos(17 * phase)
    z = np.sin(31 * phase)
    data = 0.3 * x + fault * z + rng.normal(size=n)
    a_hat = x @ data / (x @ x)
    residual = data - a_hat * x
    q = residual @ residual
    return chi2.sf(q, n - 1)

for trials in (2000, 8000, 32000):
    null = np.array([one_trial() for _ in range(trials)])
    faulty = np.array([one_trial(fault=0.30) for _ in range(trials)])
    print(trials, np.mean(null < 0.05), np.mean(faulty < 0.05))

Save the block as residual_null.py and run python residual_null.py in the pinned environment. The second printed column is the null false-alarm fraction; it should converge toward $0.05$ within its binomial standard error $\sqrt{0.05(0.95)/N_{\rm trials}}$. The third is detection power for the omitted component and should converge to a value above the false-alarm rate, increasing with fault amplitude or sample count. Repeat at sample counts 2048, 4096, and 8192, and with an independently generated orthogonal basis.

This benchmark checks fitting degrees of freedom, whitening normalization, false-alarm calibration, convergence with Monte Carlo size, and power against one known failure. It does not validate nonstationary detector noise. A production algorithm must add colored-noise generation, PSD uncertainty, calibration draws, waveform injections, selection, and a second independently implemented statistic. The run should fail if the null rate misses a predeclared three-standard-error band or if the fault is not detected with increasing power.

Results and failure modes

Evidence or result at the cutoff	Reproduction check	Limitation or failure mode
Peer-reviewed GWTC-3 residual and consistency methods, LVK 2023, Table I and §§IV–V provide a catalog baseline	Reanalyze released injections and event subsets	Results depend on waveform, calibration, PSD, and split implementation
Current GWTC-5 releases enlarge and revise the available event set	Pin stable release 9 and selection metadata	Submitted tests document is not yet a peer-reviewed replacement for every method
Null synthetic residuals can be calibrated exactly in the linear Gaussian benchmark	Recover uniform tail probabilities across resolutions	Passing the toy benchmark does not establish real-noise adequacy

The independent scientific check is cross-pipeline recovery: a residual morphology or parameter discrepancy should be reproduced with a second noise representation and waveform family, and should scale as a specified alternative predicts. Sensitivity that appears only under one PSD, sampler, or split is a diagnosed failure mode, not a gravity signal.

Check your understanding

Why is subtracting the maximum-likelihood waveform and comparing $r^Tr$ with $\chi^2_N$ incorrect in the benchmark above?

Solution

The amplitude is estimated from the same $N$ samples, so fitting projects out one noise direction. The residual therefore has $N-1$ independent Gaussian degrees of freedom, not $N$. In addition, $r^Tr$ follows the stated chi-square law only after correct whitening and with a fixed noise variance. Therefore the benchmark uses $\chi^2_{N-1}$ and checks its tail probabilities over repeated injections. In a real analysis, estimated PSD and calibration parameters consume or modify further effective degrees of freedom, which is why end-to-end simulation is safer than inserting an informal count.

References

• Gravitational Wave Open Science Center (2026), “GWTC-5.0,” official catalog and data-release documentation, stable parameter-estimation release 9. Data notes.
• LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration (2023), “Tests of general relativity with GWTC-3,” Physical Review D 107, 022005. LVK 2023, Table I and §§II–VIII.
• LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration (2026), “GWTC-5.0: Tests of General Relativity,” submitted manuscript, arXiv:2607.19293, LIGO DCC P2500781-v17. LVK 2026, Table 1 and §§2–7.

Further reading

• Yunes, N., Siemens, X., and Yagi, K. (2025), “Gravitational-wave tests of general relativity with ground-based detectors and pulsar-timing arrays,” Living Reviews in Relativity 28, 3. Yunes, Siemens, and Yagi 2025, §§4.3.7 and 4.3.9.