Interpreting Binary Black-Hole Catalogs

A binary-black-hole catalog supports a population conclusion only after the proposed source model predicts the selected, uncertain catalog and survives plausible alternatives. A fit to posterior medians, a visually striking corner plot, or a Bayes factor between two rigid models is insufficient. The necessary checks join catalog provenance, contamination, event likelihoods, injection-calibrated selection, posterior prediction, model criticism, and sensitivity to waveform and population assumptions.

Helpful background. Catalog Construction and Selection Effects defines versioned samples and injections. Source Populations and Hierarchical Inference develops rates, population distributions, and mixture limits.

Start with the exact catalog

A reproducible claim names the catalog and minor version, candidate inclusion rule, pipeline union, false-alarm or $p_{\rm astro}$ treatment, observing epochs, data-quality and calibration releases, event posterior products, waveform set, and population injection campaign. “All binary black holes” is not a sample definition.

The selection rule must match the likelihood. If marginal candidates are retained, terrestrial contamination belongs in a signal–noise mixture. If a high-purity threshold is imposed, the corresponding injection recovery must use that threshold and pipeline logic. Published $p_{\rm astro}$ values are conditional on a signal population and background model; treating them as immutable probabilities can create circular population inference.

At the cutoff 2026-08-18, the official reference is GWTC-5.0 and its associated methods, observations, event/posterior, and injection products LVK 2026, methods §§2–5 and GWOSC 2026, GWTC-5.0 data release. Later catalog versions must not be mixed silently with these selections.

Predict in the observed space

For hyperparameters $\Lambda$, the intrinsic posterior predictive distribution is

$$p(\theta_{\rm new}\mid\{d_i\}) =\int d\Lambda\, p(\theta_{\rm new}\mid\Lambda) p(\Lambda\mid\{d_i\}).$$

Catalog checking usually needs a detected predictive distribution,

$$p(\theta\mid{\rm det},\Lambda) =\frac{p_{\rm det}(\theta)p(\theta\mid\Lambda)} {\alpha(\Lambda)},$$

then simulated measurement noise and the same summary procedure used on real events. Comparing an intrinsic curve directly with noisy detected medians mixes three different distributions.

A posterior predictive check chooses a discrepancy $T$, simulates selected mock catalogs, and asks whether $T_{\rm obs}$ is typical. Useful summaries include joint mass-pairing structure, effective-spin asymmetry, redshifted mass–distance correlation, outlier counts, and event-level predictive likelihood. A model can pass the one-dimensional $m_1$ distribution while failing $p(m_1,q,\chi_{\rm eff},z)$.

Because the data help fit the model, posterior predictive $p$-values are not generally uniform calibration tests. Simulation-based calibration and held-out prediction answer different questions and should accompany them.

Cross-model checks

Every scientific feature should be tested against a model capable of erasing or relocating it:

• peaks against splines, Gaussian processes, or autoregressive processes;
• gaps against smooth tapers and partial depletions;
• spin-tilt peaks against flexible orientation densities and alternative spin measures;
• subpopulations against one correlated, nonfactorized distribution;
• redshift evolution against mass–redshift correlations;
• formation channels against multiple forward libraries with physical uncertainty.

Bayes factors compare only the specified models and prior measures. Information criteria and maximum likelihood emphasize different complexity penalties. Cross-validation asks whether added structure predicts held-out events. Agreement across these tools is stronger than any one number; disagreement is information about model fragility.

Results and interpretations for binary black holes

The GWTC-5.0 population analysis uses multiple parametric and weakly parameterized models and reports structured mass, spin, pairing, and redshift behavior. The robust level is broader than the most detailed fitted components LVK 2026, §§3–10.

Claim class	Presently supported level	Required caution
Mass spectrum	Broad peaks or slope changes survive several models	Sharp edges and feature counts vary
Spins	Effective-spin asymmetry and nonzero spin support survive	Component tilts and transition components are model dependent
Redshift	The source-frame merger rate rises over the informative range	High-redshift shape trades against mass evolution and selection
Formation	More than one physical pathway is plausible and simple one-channel stories are strained	Statistical components do not uniquely label channels

A proper evidence matrix includes negative results: models that do not require a sharp gap, tilt peak, eccentric fraction, or unique channel; injection studies that reveal bias; and event reanalyses sensitive to waveform or noise. Absence of a fitted feature in one flexible model is not proof of absence, just as its presence in one parametric model is not proof of reality.

Numerical and observational systematics

Population posteriors are Monte Carlo calculations built from posterior and injection samples. Report effective sample sizes across hyperparameter space, repeat unstable integrals with larger samples, and include their uncertainty. Apparent narrow peaks can be numerical artifacts when a few weights dominate.

Event likelihood systematics can accumulate coherently. Reweighting many events analyzed with the same imperfect waveform does not average that bias away. Calibration, PSD, glitch treatment, redshift cosmology, event priors, and waveform modes should be varied where capable of changing the claim.

Catalog heterogeneity matters too. Different observing epochs and pipelines have different sensitivity and background. Selection should be summed over disjoint configurations, not represented by one average detector.

A minimum acceptance suite

Before accepting a BBH population conclusion, require:

1. exact catalog and data-product provenance;
2. a generative likelihood with contamination and selection treated consistently;
3. posterior-prior and injection-weight support diagnostics;
4. selected mock-catalog recovery with calibrated uncertainty;
5. multidimensional posterior predictive checks;
6. at least one credible alternative population family;
7. waveform, noise, cosmology, threshold, and outlier stress tests as relevant;
8. leave-one-out or held-out predictive analysis;
9. an evidence cutoff and a concrete trigger for review;
10. a claim phrased no more strongly than the checks support.

This suite does not guarantee the true population family is present. It makes failures visible and keeps observation separate from causal interpretation.

Exercises

1. The wrong predictive curve. Why should intrinsic $p(m_1\mid\Lambda)$ not be overplotted directly on a histogram of detected posterior medians as a goodness-of-fit test?

Solution

The histogram includes mass-dependent selection, event measurement error, interim priors, and point-summary distortion, whereas the curve is an intrinsic source density. A valid comparison draws intrinsic sources, applies the catalog selection, simulates event likelihoods, and computes the same summaries—or compares the data through the hierarchical likelihood directly.

2. A successful one-dimensional check. A model reproduces the detected primary-mass distribution but fails the observed mass-ratio distribution at high mass. Is the mass model validated?

Solution

No. The factorization or pairing model is misspecified in a direction relevant to interpreting the primary-mass feature. Since selection also depends on mass ratio, the one-dimensional agreement may be compensating errors. The joint selected distribution must be repaired before assigning physical meaning to the mass feature.

References

• Gravitational Wave Open Science Center. “GWTC-5.0 Data Release.” 2026. Data and documentation.
• LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration. “GWTC-5.0: Methods for Identifying and Characterizing Gravitational-Wave Transients.” arXiv:2605.27224 (2026). Open PDF.
• LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration. “GWTC-5.0: Population Properties of Merging Compact Binaries.” arXiv:2605.27226 (2026). Open PDF.

Further reading

• Callister, Thomas A., and Will M. Farr. “Parameter-Free Tour of the Binary Black Hole Population.” Physical Review X 14 (2024): 021005. DOI. Open PDF.
• Mandel, Ilya, Will M. Farr, and Jonathan R. Gair. “Extracting Distribution Parameters from Multiple Uncertain Observations with Selection Biases.” Monthly Notices of the Royal Astronomical Society 486 (2019): 1086–1093. DOI. Open PDF.