Posterior Predictive and Residual Checks

Parameter estimation answers questions inside a model; residual and posterior-predictive checks ask whether that model can reproduce relevant features of the data. A visually quiet residual is useful but not sufficient: conditioning, parameter fitting, detector coherence, calibration uncertainty, and the chosen discrepancy all affect its reference distribution. This page constructs whitened and network residual checks, derives the posterior-predictive comparison, and states how to reproduce and interpret an event-level adequacy test without turning it into a posterior probability for the model.

Required background. Probability and Statistical Inference supplies conditional simulation, chi-square statistics, p-values, and Monte Carlo uncertainty.

Helpful background. Gravitational Radiation and Waveforms supplies detector responses and waveform families whose adequacy is being tested.

Residuals in the likelihood geometry

For detector $I$ and parameters $\theta$, define

$$r_I(t;\theta)=d_I(t)-h_I(t;\theta).$$

Under a fixed stationary Gaussian likelihood with known one-sided PSD,

$$Q(\theta) =\sum_I(r_I(\theta)\mid r_I(\theta))_I$$

is the squared norm of the whitened residual. If $k$ independent real coefficients are tested at the true fixed parameter, then $Q\sim\chi^2_k$. In practice, windowing correlates bins, PSD is estimated, and $\theta$ is fitted to the same data. The effective degrees of freedom are therefore not obtained by counting plotted pixels and subtracting an arbitrary number of parameters.

Useful residual summaries include time-domain outliers, frequency-binned energy, autocorrelation, line occupancy, time–frequency clusters, cross-detector coherence, and matched filtering against omitted waveform components. Each targets a particular failure. Passing one summary does not imply the full residual distribution is correct.

Coherent and null residuals

At a fixed sky position, let $\mathbf r(f)$ be the whitened residual vector across detectors and let $F(f)$ be the whitened network response to the modeled polarizations. When $F^\dagger F$ is invertible, define

$$P=F(F^\dagger F)^{-1}F^\dagger, \qquad P_\perp=I-P.$$

The coherent residual $P\mathbf r$ lies in the gravitational-wave response subspace, while the null residual $P_\perp\mathbf r$ is orthogonal to it. Excess coherent residual power can indicate waveform incompleteness, an unmodeled polarization within the response, or a coherent environmental feature. Excess null power can indicate detector inconsistency, glitches, calibration error, or a sky-response mismatch.

This interpretation is conditional. A poorly conditioned network response, uncertain sky position, frequency-dependent calibration, or correlated detector noise can mix the two energies. Sky and waveform parameters chosen from the same data also change the null distribution, so off-source or replicated analyses should repeat the full fitting step.

Posterior-predictive replication

Given posterior

$$p(\theta\mid d) \propto p(d\mid\theta)\pi(\theta),$$

the posterior-predictive distribution for a replicated dataset is

$$p(d^\mathrm{rep}\mid d) =\int p(d^\mathrm{rep}\mid\theta) p(\theta\mid d)\,d\theta.$$

Choose a discrepancy $T(d,\theta)$ before examining the result. A posterior-predictive tail is

$$p_\mathrm{ppc} =P\!\left[ T(d^\mathrm{rep},\theta) \ge T(d,\theta) \mid d \right].$$

It can be estimated by drawing $\theta_s$ from the posterior, simulating $d_s^\mathrm{rep}$ with the same detector, calibration, PSD, sampling, and selection model, and comparing paired discrepancies. The parameter-dependent form uses the same $\theta_s$ for observed and replicated data.

This tail is not generally uniform under the model because the observed data inform both the posterior and the observed discrepancy. Posterior-predictive p-values can concentrate near the middle of the unit interval and have limited power against some discrepancies. They are model-checking summaries, not frequentist discovery p-values and not posterior probabilities that the waveform is correct. Parameter-dependent discrepancies and their paired simulation are developed in Gelman, Meng, and Stern 1996, §2.2, PDF. For a distinct population application, Thrane and Talbot 2019, Appendix D, PDF constructs the predictive distribution of the next source’s parameters.

Choose explicitly whether replications represent a new experiment with fresh noise, a new detected event, or repeated fitting of the present design. These ensembles differ. A check based on a best-fit residual requires refitting each replicated dataset; the paired discrepancy above instead uses the same posterior draw in both terms. Mixing the two procedures changes the reference distribution.

Choosing a discrepancy without hiding failure

A discrepancy should connect to a plausible scientific failure:

• high-frequency residual power tests missing merger or ringdown structure;
• coherent residual signal-to-noise ratio tests unmodeled network-consistent strain;
• null energy tests detector inconsistency;
• residual autocorrelation tests the assumed covariance;
• time–frequency cluster energy tests transient artifacts; and
• parameter recovery under waveform alternatives tests model-induced bias.

An omnibus statistic may have little power against a localized defect. Conversely, scanning many discrepancies and reporting only the smallest tail creates a trials problem. Predeclare a compact set, show the full set, and calibrate any combined rule by repeating the same scan on replicated data.

Replications must reflect the scientific null. Gaussian draws from a fitted PSD test the Gaussian likelihood. Off-source real segments test empirical detector noise but may differ in state. Signal injections with an alternative waveform test robustness to a specified modeling error. These are distinct ensembles and should not be pooled under one label.

A fixed-parameter analytic control

Let four independent whitened residual coefficients be

$$\mathbf r=(1,-1,2,0).$$

Their energy is

$$Q=\sum_{j=1}^4r_j^2=6.$$

For four independent standard-normal coefficients at a prespecified parameter,

$$P(\chi^2_4\ge6) =e^{-3}(1+3) =0.199148\ldots .$$

This exact survival probability checks whitening, energy normalization, and tail evaluation. It is not valid after choosing the loudest time–frequency region, fitting the waveform to those four coefficients, estimating their variance from the same samples, or selecting this discrepancy because it looked unusual. A replicated pipeline must include each such operation.

Observational evidence and reproduction

At the evidence cutoff of 2026-08-18, the GWTC-5.0 methods paper documents the data, waveform, calibration, likelihood, and sampler settings that define O4b characterization LVK Collaboration 2026, §§4–5, PDF. The accompanying current tests-of-general-relativity analysis includes a versioned coherent residual test LVK Collaboration 2026, §4, PDF. These are release-specific observational analyses; no event residual or tail value is reproduced here.

A reproducible residual check pins the event and product version, GWOSC or controlled strain hashes, detectors and GPS interval, calibration and uncertainty draws, quality masks and gates, sample rate, window, frequency band, PSD data and estimator, waveform and likelihood commits, prior files, posterior-sample file and weights, sky and polarization convention, discrepancy code, random seeds, number of replications, and the rule for failed simulations. If the event or statistic was selected from a larger catalog, record that selection and reproduce it in the null ensemble.

Report the observed discrepancy, full replicated distribution, Monte Carlo uncertainty, and results under alternative PSD, calibration, waveform, glitch, and data-segment models. Inspect both coherent and detector-specific residuals. A result that changes under plausible conditioning choices is a model-sensitivity finding, not a stable anomaly.

Common pitfalls

Subtracting one best-fit waveform is not posterior prediction. A best-fit residual ignores parameter uncertainty. Predictive checks average paired replications over the posterior or explicitly state a plug-in approximation.

A small upper-tail probability does not identify the cause. Excess residual power can arise from waveform error, noise misspecification, calibration, glitches, or selection. Targeted alternatives and detector comparisons are needed. An unusually small residual can also signal overfitting or overestimated noise, which an upper-tail check alone may miss.

A moderate p-value does not validate the whole model. It says the chosen discrepancy is not unusual in the chosen replicated ensemble. Other scientifically relevant failures may be invisible to it.

Check your understanding

For the four-coefficient control above, verify the tail probability. Then suppose the residual vector was formed only after fitting one unconstrained linear amplitude to those same four coefficients. Why can the $\chi^2_4$ reference no longer be used unchanged?

Solution

For a chi-square variable with four degrees of freedom, the survival function is

$$P(\chi^2_4\ge q) =e^{-q/2}\left(1+\frac q2\right).$$

At $q=6$ this gives $4e^{-3}=0.199148\ldots$. If one linear amplitude is estimated from the same Gaussian coefficients, the fit projects out one data direction. Under the exact fixed-design linear model, the remaining residual energy has three rather than four degrees of freedom. In nonlinear waveform inference with bounds and uncertain covariance, even that simple subtraction may fail, so the safest check repeats the complete fit for every replicated dataset.

References

• Gelman, Andrew, Xiao-Li Meng, and Hal Stern. “Posterior Predictive Assessment of Model Fitness via Realized Discrepancies.” Statistica Sinica 6 (1996): 733–760; discussion and rejoinder, 760–807. Journal record. Open PDF.
• LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration. “GWTC-5.0: Methods for Identifying and Characterizing Gravitational-Wave Transients.” arXiv:2605.27224 [gr-qc] (2026). Open PDF.
• LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration. “GWTC-5.0: Tests of General Relativity.” arXiv:2607.19293 [gr-qc] (2026). Open PDF.
• Thrane, Eric, and Colm Talbot. “An Introduction to Bayesian Inference in Gravitational-Wave Astronomy: Parameter Estimation, Model Selection, and Hierarchical Models.” Publications of the Astronomical Society of Australia 36 (2019): e010. DOI. Open PDF, corrected arXiv v9.