Physics-Informed Constraints and Symmetries

Physics can enter a learned model as an exact representation, a symmetry-equivariant architecture, a projected output, or a soft penalty on equations and constraints. These choices are not interchangeable. A small training residual may coexist with a large solution error when the differential operator is ill-conditioned, boundaries are weakly enforced, or collocation misses a feature. Conversely, an exactly enforced approximate equation can make a model less accurate for the physical system it is meant to represent.

Required background. Probability and Inference for Gravity supplies held-out testing, calibration, and comparison across random training seeds.

Helpful background. Formulations of General Relativity supplies the continuum constraints, gauge choices, and symmetries being imposed; Differential Equations for Relativity supplies residual, boundary, and conditioning analysis; Computational Relativity supplies independent solvers and reproducible numerical benchmarks.

Four ways to encode physics

Let $u_\theta$ be a learned field and $\mathcal F[u]=0$ the intended continuum equation with boundary operator $\mathcal B[u]=g$. A common physics-informed objective is

$$L(\theta)= w_dL_{\rm data} +\frac{w_F}{N_F}\sum_{i=1}^{N_F} \|(\mathcal F u_\theta)(x_i)\|^2 +\frac{w_B}{N_B}\sum_{j=1}^{N_B} \|(\mathcal B u_\theta)(y_j)-g(y_j)\|^2.$$

Automatic differentiation evaluates the derivatives at collocation points. The loss depends on point distribution, nondimensionalization, derivative precision, term scaling, weights, optimizer, and network spectrum. It is an optimization problem, not a weak or strong solution theorem.

Physics can be imposed at increasing levels of rigidity:

1. data augmentation: include symmetry-transformed examples; the model may still violate the symmetry between examples;
2. soft penalty: add residuals to the loss; accuracy depends on weights and optimization;
3. equivariant parameterization: design the map so a group action on inputs produces the correct action on outputs;
4. hard construction or projection: represent outputs in variables that satisfy a relation identically, or project onto a constraint surface.

A simple hard boundary construction illustrates the difference. On $0\le x\le1$ with homogeneous Dirichlet data, write $u_\theta(x)=x(1-x)N_\theta(x)$. Both endpoint values vanish for every network parameter, but the differential equation still needs to be solved. For an exact reflection symmetry $f(-x)=f(x)$, the construction $[N_\theta(x)+N_\theta(-x)]/2$ enforces evenness; a loss penalty only encourages it.

Hard enforcement is powerful only when the encoded relation is exact for the target and the projection is stable. A divergence-free magnetic representation is helpful; a symmetry that the dataset breaks through gauge, boundary, or preprocessing choices can be harmful. Always test the raw and constrained outputs separately.

Raissi, Perdikaris, and Karniadakis present the forward and inverse PINN framework in Raissi et al. 2019, §§ 1.1–3 and § 4. Krishnapriyan and collaborators show that differential-operator losses can be poorly conditioned and hard to optimize even when a network is expressive enough in Krishnapriyan et al. 2021, §§ 3–5. Both results matter: the method is expressive, and the optimization can still fail.

Residual is not solution error

For a linearized problem $Lu=f$, an error $e=u_\theta-u$ satisfies $Le=r$, where $r=Lu_\theta-f$. Formally,

$$\|e\|\le\|L^{-1}\|\,\|r\|.$$

A small residual controls error only with an appropriate norm, correct boundary data, and a bounded inverse over the relevant function space. Near a small eigenvalue, for a nonnormal operator, or with a boundary layer missed by collocation, $\|L^{-1}\|$ can be large. Evaluating $r$ at the training points can also miss oscillations between them.

For a concrete conditioning example, consider

$$L_\varepsilon=-\frac{d^2}{dx^2}-(\pi^2-\varepsilon), \qquad u(0)=u(1)=0,\qquad 0\lt\varepsilon\ll1.$$

The exact solution of $L_\varepsilon u=0$ is $u=0$. The candidate $u_\theta=\sin(\pi x)$ satisfies both boundary values exactly yet has residual $r=\varepsilon\sin(\pi x)$. Its $L^2$ error is $1/\sqrt2$, while the residual norm is $\varepsilon/\sqrt2$. A residual of order $10^{-8}$ can therefore coexist with order-one solution error when $\varepsilon=10^{-8}$. The inverse norm on this mode is $1/\varepsilon$; no improvement in collocation can remove that conditioning.

The independent check therefore uses a dense grid or separate quadrature, finite differences or spectral derivatives not shared with automatic differentiation, and direct comparison with an exact or conventionally converged solution. It reports solution error, boundary error, constraints, conserved quantities, and cost in addition to residual.

The same distinction applies to symmetry penalties. An average equivariance loss can be small while a boundary region violates the transformation badly. Measure the maximum residual and its parameter location, and include group elements not used in augmentation.

Comparison of methods and results

The evidence below was checked through 2026-08-18.

Position	Evidence in its tested regime	Boundary of the claim
established baseline	spectral, finite-difference, continued-fraction, and reduced-order solvers provide converged references for many constraint and perturbation problems	their cost and implementation complexity motivate alternatives but remain comparison standards
positive GR evidence	Cornell et al. obtained black-hole quasinormal frequencies with percentage deviations below $0.01\%$ in selected cases	training was slower than continued fractions and WKB; the authors did not recommend the approach on overall performance
general negative evidence	PINN losses can become ill-conditioned and optimization can fail despite adequate network expressivity	curricula or sequence-to-sequence reformulations improve some tested problems, not all relativistic systems
current binary-initial-data evidence	the inspected arXiv v1 by Zhou et al. compares a guided PINN with TwoPunctures and reports relative $L^2$ errors roughly $0.8\%$–$8.9\%$ across its examples	residual reduction can decouple from global accuracy, cases need tuning, and one reported comparison was about $25$ times slower
not established	superior cost at matched strong-field accuracy, reliable full-Einstein evolution, or generalization outside the trained geometry and boundary setup	each requires independent multidomain benchmarks and held-out physical tests

The positive and performance evidence for quasinormal modes is in Cornell et al. 2022, §§ III–VI and Tables I–V. The binary-constraint source inspected here is Zhou et al. 2026, arXiv v1, § II, § III with Table I and Figs. 2–7, and § IV with Table II and Figs. 8–10; its record says accepted for Physical Review D, but the comparison here is limited to the inspected v1 text rather than an uninspected journal revision.

The favorable position is that physics terms can regularize sparse-data learning and solve selected relativistic problems accurately. The competing position is that conditioning, residual–error separation, per-case tuning, and inferior wall time currently prevent a general replacement claim.

Two-part relativistic benchmark

The first part uses a versioned nonprecessing compact-binary waveform family. Split parameter points before training into interpolation, boundary, contiguous-hole, and out-of-domain sets. Compare three otherwise matched coefficient surrogates: unconstrained, soft symmetry-penalized, and one parameterized to transform exactly under simultaneous exchange of the masses, spins, orbital phase, and waveform frame. Symmetric mass ratio alone is insufficient: with $\delta=(m_1-m_2)/(m_1+m_2)$ and $\chi_a=(\chi_1-\chi_2)/2$, both change sign under label exchange, so $\delta\chi_a$ is invariant while $\chi_a$ alone is not. Preserve any mode-dependent transformation rather than imposing scalar invariance on every mode. Report basis error, median/95th/maximum waveform mismatch, phase and amplitude error, exchange-symmetry residual, and variation over at least five seeds.

Increase training coverage, model capacity, optimizer accuracy, loss weights, collocation density where used, and arithmetic precision separately. A representative acceptance rule is maximum in-domain mismatch below $10^{-3}$ and hard-symmetry residual below $10^{-12}$ in binary64. The soft model must show its actual worst residual; a lower data loss does not excuse a failed symmetry threshold. The adverse case withholds a strip touching equal mass and checks whether a shortcut based on parameter ordering appears.

The second part solves a fixed Bowen–York binary-black-hole Hamiltonian-constraint problem with all puncture masses, momenta, spins, locations, domain map, regularization, and boundary conditions declared. Compare the learned solution with a conventionally converged spectral calculation at three reference resolutions. Evaluate on an independent dense grid:

$$\frac{\|u_\theta-u_{\rm ref}\|_2}{\|u_{\rm ref}\|_2}, \qquad \|\mathcal H[u_\theta]\|_2, \qquad \|\mathcal H[u_\theta]\|_\infty,$$

as well as boundary error, ADM mass, and pointwise behavior near both punctures. Perturb or remove one term from the physics loss as a mutation test; the independent residual and global quantities must reveal the error. Match the classical and learned methods at the same solution-error tolerance before comparing wall time and total accelerator resources.

The environment record pins waveform and puncture data revisions, preprocessing, splits, equation and boundary conventions, architecture, loss weights, collocation generator, optimizer and schedule, seeds, precision, automatic-differentiation and linear-algebra libraries, compiler or interpreter, operating system or container, accelerator, and reference-solver settings. Preserve checkpoints only as intermediate products; the finite evidence is the independently recomputed comparison.

Reproducing the calculation and recognizing failure

Failure modes include a low training residual with high independent solution error, collocation that misses puncture or boundary structure, loss terms with incompatible scales, seed-sensitive convergence, a soft symmetry that fails at a parameter edge, and an exact constraint that encodes an approximate or wrong physical equation. A method also fails the performance claim if a classical solver reaches the same accuracy faster or more robustly.

A clean reconstruction verifies all inputs; rebuilds each waveform model and constraint solver over every seed; sweeps data, collocation, capacity, optimization tolerance, and precision; evaluates held-out waveforms and independent PDE residuals; runs the omitted-term and boundary counterexamples; compares the classical solver at matched accuracy; and emits actual errors, constraint norms, costs, thresholds, and failures. The result is conditional on the fixed equations, domains, data, and hardware.

Review is triggered by a peer-reviewed revision that changes the 2026 binary-constraint results, an independent reproduction, a multidomain or full-Einstein strong-field benchmark, a method that changes the matched-accuracy classical comparison, a new residual–error counterexample, or a changed architecture, data domain, or constraint definition.

Check your understanding

A PINN’s Hamiltonian-constraint residual is $10^{-8}$ at every training collocation point, but an independent spectral-grid evaluation finds relative solution error $5\%$ and a boundary residual of $10^{-2}$. Has the physics term verified the solution?

Solution

No. The training residual shows only that the chosen optimizer and collocation set made that sampled loss small. The boundary failure and independent solution error falsify the accuracy claim. Densify and adapt collocation, nondimensionalize and rebalance terms, evaluate derivatives independently, and compare against the converged reference. Acceptance must be based on the independent errors and physical quantities, not the training loss.

Continue with Out-of-Distribution and Uncertainty Tests.

References

• Cornell, Alan S., Anele Ncube, and Gerhard Harmsen. “Using Physics-Informed Neural Networks to Compute Quasinormal Modes.” Physical Review D 106 (2022): 124047. arXiv:2205.08284. DOI.
• Krishnapriyan, Aditi S., Amir Gholami, Shandian Zhe, Robert M. Kirby, and Michael W. Mahoney. “Characterizing Possible Failure Modes in Physics-Informed Neural Networks.” In Advances in Neural Information Processing Systems 34. 2021. arXiv:2109.01050.
• Raissi, Maziar, Paris Perdikaris, and George Em Karniadakis. “Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations.” Journal of Computational Physics 378 (2019): 686–707. DOI.
• Zhou, Yu-Chen, et al. “Solving Hamiltonian Constraint Equation with Physics-Informed Neural Networks.” arXiv:2607.06002v1, submitted 7 July 2026. arXiv.