Model Selection and Bayesian Evidence

Model appraisal asks whether extra freedom earns its keep through better prediction, not merely whether it lowers a best-fit $\chi^2$. Bayesian evidence, information criteria, cross-validation, and posterior predictive checks formalize different questions and can legitimately disagree. As of 2026-08-19, no single scalar criterion supplies a model-independent ranking of cosmologies: every conclusion depends on the candidate model class, likelihood release, prior measure, parameter regime, and systematic treatment.

Required background. Probability and Statistical Inference supplies normalized probability models, priors, predictive distributions, and Monte Carlo diagnostics.

Helpful background. Einstein Equations supplies the gravitational hypotheses being compared; Relativistic Matter and Compact Objects supplies competing matter models; Relativistic Cosmology supplies observables and physical parameterizations; and Computational Relativity supplies numerical convergence and simulation-based calibration.

Marginal likelihood averages predictive fit over a prior

For model $M_j$ with parameters $\boldsymbol\theta_j$, its Bayesian evidence is

$$Z_j=p(D\mid M_j) =\int d\boldsymbol\theta_j\, p(D\mid\boldsymbol\theta_j,M_j) \pi(\boldsymbol\theta_j\mid M_j).$$

Posterior model odds obey

$$\frac{p(M_1\mid D)}{p(M_0\mid D)} =\frac{Z_1}{Z_0} \frac{p(M_1)}{p(M_0)}.$$

Thus the Bayes factor $B_{10}=Z_1/Z_0$ updates, but does not replace, model prior odds. Both likelihood and parameter priors must be normalized. A likelihood constant harmless for parameter estimation can change $Z_j$ when the data dimension, covariance, or normalization differs between models.

Evidence implements an Occam factor. Consider $M_0$ with $\psi=0$ and an extension $M_1$ whose likelihood near its maximum is Gaussian with width $s$, while $\psi$ has a uniform prior of width $\Delta\gg s$. Away from prior boundaries,

$$B_{10}\simeq \exp(\Delta\ln\mathcal L_{\max}) \frac{\sqrt{2\pi}\,s}{\Delta},$$

where $\Delta\ln\mathcal L_{\max}=\ln\mathcal L(\widehat\psi)-\ln\mathcal L(0)$. The first factor rewards fit; the second penalizes prior volume that made predictions not supported by the data. Trotta 2008, §§4.1–4.4 derives this balance and explains why unconstrained added parameters are not penalized in the same way as well-measured but wasted prior directions.

This one-dimensional estimate assumes no additional nuisance integration, or that the common-parameter integrals cancel. In a cosmological extension, correlations can change those integrals, so a profile-likelihood width alone does not generally determine the evidence. Conversely, if an added parameter is completely unmeasured and the likelihood is independent of it everywhere on its normalized prior, its integral contributes exactly one. Extra parameter count by itself is therefore not a Bayesian penalty.

Prior choice and parameterization are part of the model

A prior uniform in $w$ is not uniform in a nonlinear function $f(w)$. Under $\phi=f(\theta)$,

$$\pi_\phi(\phi)= \pi_\theta(\theta(\phi)) \left\lvert\frac{d\theta}{d\phi}\right\rvert.$$

Bayesian evidence is invariant under a coordinate change only when the same measure is transformed with its Jacobian. Declaring a new flat prior defines a new model. Physically motivated priors may encode stability, causality, an effective-field-theory domain, or a population scale. Reference or objective constructions pursue other desiderata. Neither label eliminates the need to publish bounds and test sensitivity.

For a meaningful sensitivity analysis, vary defensible prior families and widths before seeing the model-comparison result, report the induced change in $\ln B$, and identify whether the posterior reaches a boundary. An extension whose preference disappears under small, physically reasonable changes is not robustly selected. An arbitrarily broad prior can also manufacture support for the simpler model.

Different criteria answer different questions

Criterion	Quantity compared	What it can establish	Principal limitation
Bayesian evidence	Prior-averaged normalized likelihood	Relative support among declared models and priors	Sensitive to prior measure and likelihood normalization
AIC	$-2\ln\mathcal L_{\max}+2k$	Approximate expected predictive loss under regular conditions	Counts nominal parameters and does not encode physical prior volume
BIC	$-2\ln\mathcal L_{\max}+k\ln N$	Large-sample approximation under restrictive regular assumptions	Ambiguous effective $N$ for correlated fields; hidden prior approximation can fail
Cross-validation	Held-out log predictive density	Out-of-sample prediction for the chosen split	Correlated sky modes or objects require blocked, not random, splits
Posterior predictive check	Replicated-data discrepancy	Absolute inadequacy in a chosen observable	A non-extreme check does not rank models or prove truth
Model averaging	Prediction weighted over models	Propagates declared model uncertainty	Inherits model-list, prior, evidence, and misspecification choices

AIC, BIC, and Bayesian evidence are not interchangeable penalties. Trotta 2008, §4.7 shows that BIC’s convenient absence of an explicit user prior conceals an approximation and that AIC and BIC penalize extra parameters even when the data do not constrain them. Cross-validation instead asks how a fitted model predicts withheld data. Posterior predictive checks ask whether replicated data resemble the observation in a prespecified aspect; they can reject every candidate.

Results and limitations through 2026-08-19

The evidence status through 2026-08-19 favors a portfolio of calibrated diagnostics. The strongest unresolved disagreement is how to weigh physically disputed prior volume against flexible late-time fits when data combinations themselves retain calibration and systematic uncertainty.

Evidence or result	Appraisal consequence	Scope, limitation, or open question
Nested-Gaussian analytic control	Separates maximum-likelihood gain from the prior-to-posterior volume ratio	Regular, unimodal approximation; boundaries and non-identifiability can invalidate it
Trotta’s Bayesian treatment	Provides normalized evidence, Bayes factors, complexity, and information-criterion comparisons	Trotta 2008, §§2–4 is methodological; a current verdict still requires versioned data and priors
Planck 2018 likelihoods	Supply a concrete high-dimensional likelihood in which model extensions change degeneracies and derived parameters	Planck Collaboration 2020, §§2 and 7 is a 2018-data example, not an unconditional ranking of later models
DESI DR2 model fits	Show that likelihood gain and inferred dark-energy preference vary with supernova sample, CMB choice, parameterization, and statistic	Abdul Karim et al. 2025, §§IV and VII, Tables V–VI reports conditional results and explicitly examines alternative systematics
2026 cross-probe review	Finds the broad $\Lambda$CDM description successful while current hints remain unsettled	Cortês, Lahav, and Liddle 2026, §§25.4–25.5 does not turn consensus utility into proof of model truth
Simulations drawn from each candidate	Calibrate false-selection rates, predictive scores, boundaries, and look-elsewhere effects	Only probes injected models and simulated systematics; shared misspecification may survive

The DESI DR2 analysis is instructive without freezing its numerical preferences here. Changing the supernova likelihood changes the inferred improvement of a time-varying dark-energy model; changing the dark-energy parameterization changes the prior geometry; and alternative CMB compressions retain different information. That pattern is evidence about conditional sensitivity, not permission to average significances or select the most favorable combination.

Competing appraisal positions

One position prioritizes Bayesian evidence with physical priors, treating predictive concentration as intrinsic to a model. A second uses reference or objective priors to reduce investigator discretion, though such priors remain problem dependent. A third emphasizes out-of-sample prediction, especially when a flexible phenomenological parameterization lacks a compelling physical prior. A fourth demands posterior predictive adequacy before any relative ranking. A fifth uses model averaging because no candidate is decisively selected.

These positions need not be mutually exclusive. A defensible appraisal can report evidence over a prior-sensitivity grid, blocked cross-validation, and prespecified predictive checks. Agreement is stronger than any one score; disagreement identifies whether prior volume, extrapolation, or absolute fit drives the conclusion.

Independent checks before interpreting a preference

First, repeat the calculation under one-to-one parameter transformations while carrying the Jacobian; the evidence must agree. Then vary physically defensible prior bounds, likelihood release, calibration nuisance model, and scale selection. Report evidence-integration uncertainty separately from scientific sensitivity.

Second, simulate data from every candidate model, not only the baseline. Apply the complete selection and fitting pipeline, including nuisance boundaries and any search over redshift bins or parameterizations. The resulting distribution calibrates Bayes factors, information criteria, and predictive scores under known truth. A statistic that cannot recover either candidate in its own favorable regime is not discriminating.

Third, withhold coherent units such as sky regions, redshift blocks, experiments, or observing epochs. Randomly splitting highly correlated modes leaks information and exaggerates predictive performance. Finally inspect posterior predictive residuals: a model can win a relative contest while fitting every scientifically important feature poorly.

Exercise: fit gain versus prior volume

An extension improves the maximum log likelihood by $\Delta\ln\mathcal L_{\max}=4.5$. Its added parameter has a Gaussian likelihood width $s=0.2$ and a uniform prior width $\Delta=4$. Estimate $B_{10}$ in the wide-prior approximation. What happens if the same analysis uses $\Delta=40$?

Solution

For $\Delta=4$,

$$B_{10}\simeq e^{4.5}\frac{\sqrt{2\pi}(0.2)}{4} \simeq 11.3.$$

The fit improvement alone is $e^{4.5}\simeq90$, but most of the wider model’s prior predictions are not supported, so the Occam factor reduces the Bayes factor. Increasing the prior width by ten while leaving the likelihood unchanged divides $B_{10}$ by ten, giving about $1.13$. The maximum likelihood and a nominal three-sigma-like fit displacement have not changed; the model-comparison conclusion has, because the prior predictive model is different.

Scope and open questions

The interpretation could change after a model achieves genuine out-of-sample predictive success across independent probes, after a principled physical prior revision changes the prior measure, or after a new likelihood produces a stable gain across calibration, systematic, parameterization, and prior choices. Any of those events could change the evidence balance.

The page compares appraisal methods and their failure modes. It does not publish undated best-fit values, endorse one current extension, or treat selection among an incomplete and possibly misspecified candidate list as proof that the winning model is true.

References

• Abdul Karim, M., et al. (DESI Collaboration). 2025. “DESI DR2 Results II: Measurements of Baryon Acoustic Oscillations and Cosmological Constraints.” Physical Review D 112: 083515. arXiv:2503.14738v3. DOI. Open PDF.
• Cortês, Marina, Ofer Lahav, and Andrew R. Liddle. 2026. “Cosmological Parameters.” In Review of Particle Physics, revised October 2025. Particle Data Group. Official PDF.
• Planck Collaboration. 2020. “Planck 2018 Results. VI. Cosmological Parameters.” Astronomy & Astrophysics 641: A6. DOI. Open PDF.
• Trotta, Roberto. 2008. “Bayes in the Sky: Bayesian Inference and Model Selection in Cosmology.” Contemporary Physics 49: 71–104. DOI. Open PDF.