Hierarchical Tests Across Catalogs

Scrollable formula. Use Left and Right arrow keys to reveal hidden content.

Scrollable table. Use Left and Right arrow keys to reveal additional columns.

Scrollable figure. Use Left and Right arrow keys to inspect hidden content.

Combining gravitational-wave events can reveal a weak deviation that no single event resolves, but only if the events constrain a coherently defined quantity. A catalog hierarchy must preserve source dependence, event-level interim priors, detection selection, calibration shared across events, and overlap among released analyses. Multiplying marginal posteriors erases those ingredients and can turn a prior convention or changing event mix into an apparent universal law.

Helpful background. Probability and Statistical Inference supplies hierarchical measures and selection normalization; Gravitational-Wave Data Analysis and Inference supplies event and population likelihoods; Weak-Field Gravity supplies the perturbative deformation coordinates; Tests of Gravity supplies the theory map; and Black-Hole Perturbation Theory explains why remnant-mode deviations may depend on mass, spin, and mode.

A selection-normalized catalog likelihood

Let $x_i$ denote calibrated data for detected event $i$ , $\theta_i$ its source parameters, and $\delta_i$ a phenomenological gravity coordinate. Put calibration, waveform, and other catalog-wide nuisance variables in $\zeta$ . Given population hyperparameters $\Lambda$ , the likelihood conditioned on detecting $N$ events is schematically

\mathcal L(\Lambda) \propto \int d\zeta\,p(\zeta) \prod_{i=1}^{N} \frac{ \displaystyle\int d\theta_i\,d\delta_i\, p(x_i\mid\theta_i,\delta_i,\zeta) p(\theta_i,\delta_i\mid\Lambda) }{ \alpha(\Lambda,\zeta) },

where

\alpha(\Lambda,\zeta) =\int d\theta\,d\delta\, p_{\rm det}(\theta,\delta\mid\zeta) p(\theta,\delta\mid\Lambda)

is the detection efficiency at fixed shared nuisance state. If the astrophysical rate is inferred rather than conditioned away, the Poisson rate factor must be retained consistently; mixing the two conventions changes the measure. The outer integral keeps catalog-wide calibration, waveform, and population assumptions shared rather than treating them as independent event errors.

Event posteriors are often produced under interim priors $\pi_{0,i}(\theta,\delta)$ . With samples $(\theta_{is},\delta_{is})$ , the event integral, up to its $\Lambda$ -independent event evidence, is estimated by

\frac{1}{S_i}\sum_{s=1}^{S_i} \frac{p(\theta_{is},\delta_{is}\mid\Lambda)} {\pi_{0,i}(\theta_{is},\delta_{is})}.

The denominator is indispensable. This simple estimator assumes the event likelihood is the same as in the original fit, with shared nuisances fixed or consistently retained. Independently marginalizing calibration for each event and then recycling only $(\theta,\delta)$ samples does not recover the outer shared- $\zeta$ integral. A changed likelihood needs an additional likelihood ratio or a new event analysis; conditional event evidences that depend on $\zeta$ cannot be dropped from its integral.

Effective sample size, $S_{\rm eff}=(\sum_s w_s)^2/\sum_s w_s^2$ , diagnoses concentration of the importance weights $w_s=p_\Lambda/\pi_0$ . It cannot detect support that was never sampled. A hypermodel outside the interim-prior support cannot be recovered by reweighting. Mandel, Farr, and Gair give the selection-normalized construction and posterior-recycling logic Mandel, Farr, and Gair 2019, §§2–4.

A Gaussian population example

Suppose a measured deviation obeys $y_i\mid\delta_i\sim\mathcal N(\delta_i,s_i^2)$ and the underlying deviations obey $\delta_i\sim\mathcal N(m,\sigma_{\rm pop}^2)$ . In a toy sample with no selection dependence or shared systematic error, integrating over $\delta_i$ gives

y_i\mid m,\sigma_{\rm pop} \sim\mathcal N(m,s_i^2+\sigma_{\rm pop}^2).

The mean $m$ tests a systematic shift; $\sigma_{\rm pop}$ tests extra event-to-event scatter. The strict common-GR model is $(m,\sigma_{\rm pop})=(0,0)$ , with the width on a boundary. At $\sigma_{\rm pop}=0$ , the estimate of $m$ is the inverse-variance weighted mean, with variance $(\sum_i s_i^{-2})^{-1}$ . A calibration error common to all events does not average down by this law. Nor does a credible interval near $\sigma_{\rm pop}=0$ automatically give a calibrated likelihood-ratio significance: boundary and hyperprior effects matter.

What is shared across sources?

Three common hypotheses answer different questions:

A universal deformation sets $\delta_i=\delta$ for every source. It is sharp but often physically unjustified.
A population deformation draws $\delta_i$ from a distribution with mean and width, allowing intrinsic variation while testing whether zero remains typical.
A theory-linked deformation computes $\delta_i=g(\theta_i,z_i,\mathcal E_i;\lambda)$ from one coupling $\lambda$ , source parameters, redshift, and environment $\mathcal E_i$ .

A phase coefficient, damping-time shift, and polarization amplitude are not interchangeable instances of one generic “deviation.” Even the same named coefficient may be normalized differently across waveform families or valid only over a mass and frequency range. Hierarchical shrinkage can pull noisy event estimates toward the population mean; that is intended behavior, not independent evidence that each event shares the inferred value.

Selection also depends on the alternative. A non-GR waveform can change search efficiency, recovered signal-to-noise ratio, event classification, or which pipeline-quality cuts are passed. Using GR injections for $\alpha(\Lambda)$ is justified only while deviations are demonstrably too small to alter detection. Catalog and test selection are distinct: at the evidence cutoff 2026-08-19, GWTC-5 contains 390 catalog detections, while the submitted tests manuscript identifies 168 signals selected for at least one GR test and then applies narrower pipeline cuts LIGO–Virgo–KAGRA 2026, submitted P2500781-v17, Table 1 and §§2–3.

Versioned data and shared systematics

The official input baseline is GWTC-5/GWOSC stable parameter-estimation release 9 with DOI 10.7935/bk00-6a89 GWOSC 2026, official release documentation; P2600410-v1 separately supplies products for reproducing the tests-manuscript figures LIGO–Virgo–KAGRA 2026, official tests data release. Reproduction records catalog minor version, event aliases, calibrated strain and posterior file identifiers, waveform and prior definitions, selection injections, software commit, hyperprior, sampler convergence, and posterior-recycling effective sample sizes. The selection function must use the same data-quality epoch and search threshold as the event set.

Correlated calibration, waveform error, cosmology, and detector-state effects should enter as shared latent variables or covariance, not be added independently in quadrature for every event. Analyses that reuse one event or posterior product cannot be treated as independent catalog evidence. A result combined across coefficient, residual, and ringdown pipelines requires their joint likelihood; multiplying their Bayes factors would double count the same strain.

The peer-reviewed GWTC-3 study demonstrates both restricted common-deviation and hierarchical population combinations and states their event selections LIGO–Virgo–KAGRA 2023, §II.C and §§V–VIII. Current submitted GWTC-5 combinations use analysis-specific hierarchies; their labels and priors must be read from the relevant sections rather than assumed from earlier catalogs.

Results and limitations through 2026-08-19

Evidence or result	Position supported	Limitation or uncertainty
Selection-normalized hierarchy	Combines weak information without treating detected events as an unbiased population	Requires reliable efficiency over the same source and deviation domain
Peer-reviewed GWTC-3 combinations	Found no established shared deviation in tested coordinates	O3 selection, interim priors, waveform systematics, and finite events delimit the claim
Submitted GWTC-5 combinations	Improve precision for selected current tests using more events	Submitted status; 168 test-selected signals are not all 390 detections and cuts differ by pipeline

The conservative position is that tested catalog-level coordinates remain compatible with their GR nulls under the published and submitted models. A competing interpretation could arise if a source-dependent effect is incorrectly forced to be universal, or if a hyperprior excludes zero-volume structures differently from the event priors. Conversely, a broad population model can dilute a truly universal small shift.

An adversarial independent check generates detected catalogs with known universal, population-varying, and source-scaled deviations, passes them through the actual selection, and verifies coverage under alternate interim priors and waveform families. It also duplicates one event deliberately to demonstrate the signature of double counting. Until those controls reproduce the hyperparameters and false-positive rate, a catalog anomaly remains unresolved and model-dependent.

What could change the conclusion. The interpretation could change after a catalog, stable posterior, calibration, or injection release changes the selected population; after a theory supplies a different source-dependent map; or when a validated joint likelihood replaces combinations that assumed independence.

Exercises

Two posterior samples have interim-prior densities $\pi_0=0.5$ and $0.25$ and hypermodel densities $p_\Lambda=0.25$ and $0.50$ . Compute their recycling estimate up to the event evidence and explain which sample dominates.

Solution

The importance weights are $0.25/0.5=0.5$ and $0.50/0.25=2$ . Their average is $(0.5+2)/2=1.25$ , which estimates the event integral divided by the evidence used to normalize the interim posterior. The second sample dominates because the hypermodel assigns it more density relative to the interim prior. This comparison also shows why omitting $\pi_0$ is wrong: it would confuse how samples were generated with support for the population model.

References

Gravitational Wave Open Science Center (2026), “GWTC-5.0 Data Release Documentation,” stable release 9 parameter-estimation, strain, and data-quality products, official release, DOI.
Ilya Mandel, Will M. Farr, and Jonathan R. Gair, “Extracting Distribution Parameters from Multiple Uncertain Observations with Selection Biases,” Monthly Notices of the Royal Astronomical Society 486, 1086–1093 (2019), DOI.
LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration, “Data Release for GWTC-5.0: Tests of General Relativity,” version P2600410-v1 (2026), official release.
LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration, “Tests of General Relativity with GWTC-3,” Physical Review D 107, 022005 (2023), DOI.
LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration, “Tests of General Relativity with GWTC-5.0,” submitted manuscript P2500781-v17 (2026), official DCC record.