Thermodynamics, Statistical Mechanics, and Information

Thermodynamics describes equilibrium through macroscopic states and allowed processes. Statistical mechanics supplies probability laws over microscopic states. Information theory asks what a probability distribution or density operator says about uncertainty, correlations, and distinguishability. The same logarithms appear in all three subjects, but the physical interpretation comes from the system, controls, observables, and operational question—not from the formula alone.

Helpful background. Calculus and Linear Algebra for Relativity provides multivariable differentiation, linear maps, bilinear forms, and eigenvalue language; the probability notation is introduced in this chapter. Variational Mechanics and Symmetries provides constrained variations and Legendre-transform language; Quantum States, Observables, and Measurement supplies state, observable, and Hamiltonian language used on the quantum pages.

Three levels of description

At the macroscopic level, an equilibrium state is specified by a complete set of variables such as $(U,V,N)$ or $(T,V,N)$. A process can connect equilibrium endpoints, but only a quasistatic process remains close enough to equilibrium to trace a path through that state space. Internal energy and entropy are state functions; heat and work describe energy transfer along a process and are not properties stored in the system.

A statistical ensemble adds microscopic states and weights. The microcanonical ensemble fixes energy, the canonical ensemble describes weak thermal contact with a reservoir, and the grand-canonical ensemble also exchanges particles. The choice records a physical coupling and a set of controls. It is not merely a choice of computational convenience.

Information measures can then be applied to those weights. Shannon or Gibbs entropy measures the spread of one discrete distribution; relative entropy compares two distributions; mutual information measures correlations. In quantum theory, the density operator replaces a classical probability vector, the von Neumann entropy depends on its spectrum, and a partial trace describes the state available to a subsystem.

The principal objects are related, but they answer different questions:

Description	Input	Central object	Question answered
Thermodynamics	equilibrium state and environmental controls	$U$, $S$, or a Legendre-transformed potential	Which state is stable and which exchanges are allowed?
Statistical mechanics	microstates, conserved quantities, and weights	$\mathcal N_E$, $Z$, or $\Xi$	Which macroscopic values and fluctuations follow from the ensemble?
Information theory	one or more distributions	$H$, $D$, or $I$	How uncertain, distinguishable, or correlated are the variables?
Quantum statistics	Hilbert space, Hamiltonian, and density operator	$\rho$, $S(\rho)$, or $D(\rho\lVert\sigma)$	What does the chosen observable algebra predict?

Throughout the chapter $k_{\mathrm B}=1$, so entropy is dimensionless and temperature has units of energy. Natural logarithms are used, and the equilibrium variational statements are restricted to $T\gt 0$. Restoring ordinary units replaces $S$ by $k_{\mathrm B}S$ wherever a statistical entropy is given as a logarithm.

The six-page path

The pages are ordered so that each new mathematical object answers a question left open by the preceding one.

1. States, Processes, and the Laws of Thermodynamics distinguishes equilibrium states from processes, derives the fundamental differential at fixed and variable particle number, and states the entropy-production form of the second law.
2. Thermodynamic Potentials and Stability performs the Legendre transforms explicitly, matches each potential to its natural controls, and checks stability through response coefficients and Hessians.
3. Statistical Ensembles and Partition Functions derives canonical weights from a reservoir, extracts means and variances from $Z$ and $\Xi$, introduces Bose–Einstein and Fermi–Dirac occupations, and explains ensemble equivalence through supporting lines.
4. Entropy, Relative Entropy, and Information compares thermodynamic, Gibbs, Shannon, and relative entropy without conflating them, then proves positivity and states data processing.
5. Fluctuations, Response, and Susceptibilities connects equilibrium covariance to static response and gives the convention-complete quantum Kubo and fluctuation–dissipation formulas.
6. Quantum Statistical Mechanics and Density Operators develops mixtures, Gibbs states, the free-energy variational principle, quantum relative entropy, partial traces, and entanglement entropy.

Readers interested mainly in classical thermodynamics can follow pages 1–3 and then the static half of page 5. Readers arriving from quantum mechanics can begin with page 3, use page 4 for relative entropy, and continue through pages 5–6. The full sequence is best when the goal is to understand which assumptions connect macroscopic stability, microscopic weighting, response, and quantum mixedness.

One mathematical chain

For an isolated system, microcanonical counting defines $S(E)=\ln\mathcal N_E(E)$. Weak thermal contact with a large reservoir then produces the canonical partition function

$$Z(\beta)=\sum_i e^{-\beta E_i}, \qquad U=-\partial_\beta\ln Z, \qquad \operatorname{Var}(E)=\partial_\beta^2\ln Z.$$

The same generating function yields the Helmholtz free energy $F=-T\ln Z$. Its derivatives reproduce the thermodynamic differential, while its curvature encodes susceptibility and stability. Tong develops this sequence from microcanonical counting through canonical and grand-canonical ensembles in Tong, current notes, §§1.2–1.4, printed pp. 2–30.

The connection is powerful but conditional. Canonical concentration around one energy requires a stable saddle; thermodynamic equivalence at a microcanonical energy requires an appropriate supporting line of the entropy density. At phase coexistence, for finite systems, or for nonadditive long-range interactions, a single Gaussian saddle can fail. Touchette separates thermodynamic, macrostate, and measure equivalence in Touchette 2015, §§III–VI.A, author-manuscript pp. 10–25.

Quantum mechanics changes the state space, not the logic of normalization. For a finite-dimensional Hamiltonian,

$$\rho_\beta=\frac{e^{-\beta H}}{\operatorname{Tr}e^{-\beta H}}, \qquad S(\rho)=-\operatorname{Tr}(\rho\ln\rho).$$

Relative entropy then proves that $\rho_\beta$ uniquely minimizes the quantum free-energy functional. A partial trace can make a subsystem mixed even when the complete state is pure, so von Neumann entropy need not represent heat or an unknown classical preparation label.

Interpretation boundaries

Thermodynamic entropy, Gibbs or Shannon entropy, relative entropy, von Neumann entropy, and a chosen coarse-grained entropy are related but not interchangeable. Black-hole entropy adds further geometric and gravitational input. This chapter does not derive black-hole mechanics, Hawking temperature, Bekenstein–Hawking or Wald entropy, generalized entropy, or any conclusion about the information problem. Those questions begin in Black-Hole Thermodynamics and Semiclassical Gravity, after the classical and quantum distinctions here are secure.

The next chapter, Fluids, Plasmas, and Kinetic Theory, turns equilibrium state variables and local distribution functions into dynamical matter models.

Review the chapter

1. Control variables. A system is held at fixed temperature and volume while it exchanges particles with a reservoir. Which potential determines equilibrium, and which second derivative gives the particle-number fluctuation?

Solution

The controls are $(T,V,\mu)$, so equilibrium minimizes the grand potential $\Omega=-T\ln\Xi$. With $\beta$ fixed,

$$\langle N\rangle=\frac{1}{\beta}\frac{\partial\ln\Xi}{\partial\mu}, \qquad \operatorname{Var}(N)=\frac{1}{\beta^2}\frac{\partial^2\ln\Xi}{\partial\mu^2}\geq0.$$

The nonnegative variance is both a fluctuation identity and a convexity check on $\ln\Xi$.

2. Entropy diagnosis. A pure bipartite quantum state has a mixed reduced state. Does the subsystem entropy necessarily equal heat divided by temperature?

Solution

No. The reduced von Neumann entropy quantifies correlations across the chosen bipartition. Calling it thermodynamic entropy additionally requires an equilibrium or controlled coarse-graining argument, a Hamiltonian and temperature, and a specified subsystem or observable algebra. The shared logarithmic form does not supply those physical conditions.

3. Ensemble diagnosis. Suppose an entropy density $s(e)$ is nonconcave at an energy $e_0$ and admits no affine supporting line there. Can any canonical inverse temperature $\beta$ select $e_0$ as its equilibrium energy?

Solution

No. A canonical equilibrium energy must maximize $s(e)-\beta e$, which is equivalent to the supporting-line inequality

$$s(e')\leq s(e_0)+\beta(e'-e_0)$$

for every allowed $e'$. If no such line exists, some other energy always gives a larger canonical exponent. The microcanonical state at $e_0$ can still be meaningful, so this is ensemble nonequivalence rather than the nonexistence of the isolated system.

References

• Tong, David. Statistical Physics. University of Cambridge Part II Mathematical Tripos lecture notes, current official undated-cover PDF, §§1.2–1.4, printed pp. 2–30. Open PDF.
• Touchette, Hugo. “Equivalence and Nonequivalence of Ensembles: Thermodynamic, Macrostate, and Measure Levels.” Journal of Statistical Physics 159 (2015): 987–1016, §§III–VI.A. doi:10.1007/s10955-015-1212-2; Open PDF.