Thermodynamics

• Yuxi Liu’s blog
  • [90] Classical Thermodynamics and Economics
  • [70] Statistical mechanics
  • [90] Analytical mechanics
• Ortega and Braun papers:
  • [80] thermodynamics as a theory of decision-making with computational costs
  • [80] information-theoretic bounded rationality
  • [80] information, utility and bounded rationality
• [75] Second Law of Thermodynamics by Eliezer Yudkowsky
• [75] An equilibrium of no free energy by Eliezer Yudkowsky [90] Generalized Heat Engine by John Wentworth
• [80] A Statistical Equilibrium Theory of Markets by Duncan Foley
• [80] Free energy principle wikipedia page
• [50] Richard Ngo’s tweet and replies thereoef
• [50] Demystifying the second law of thermodynamics by Eigil Rischel
• [20] Clarifying the free energy principle (with quotes)
• [50] Physical foundations of Landauer’s principle by Michael P Frank
• [80] Bayesian mechanics
• [40] Bayesian mechanics - map/territory
• [60] Shore & Johnson (1980): Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-Entropy
• [50] Feynman lectures on Computation

One informal way to think of homomorphisms in math is that they are maps that do not “create information out of thin air”. Isomorphisms further do not destroy information. The terminal object (e.g. the trivial group, the singleton topological space, or the trivial vector space) is the “highest-entropy state”, where all distinctions disappear and reaching it is heat death.

• Take, for instance the group homomorphism \(\phi:\mathbb{Z}^+\to\mathbb{Z}_{4}^+\). Before \(\phi\) was applied, “1” and “5” were distinguished: 2 + 3 = 5 was correct, but 2 + 3 = 1 was wrong. Upon applying this homomorphism, this information disappears — however, no new information has been created, that is: no true indinstinctions (equalities) have become false.
• Similarly in topology, “indistinction” is “arbitrary closeness”. Wiggle-room (aka “open sets”) is information, it cannot be created from nothing. If a set or sequence goes arbitrarily close to a point, it will always be arbitrarily close to that point after any continuous transformations.
• There is no information-theoretical formalization of “indistinction” on these structures, because this notion is more general than information theory. In the category of measurable spaces, two points in the sample space are indistinct if they are not distinguished by any measurable set — and measurable functions are not allowed to create measurable sets out of nothing.

(there is also an alternate, maybe dual/opposite analogy I can make based on presentations — here, the the highest-entropy state is the “free object” e.g. a discrete topological space or free group, and each constraint (e.g. \(a^5=1\)) is information — morphisms are “observations”. In this picture we see knowledge as encoded by identities rather than distinctions — we may express our knowledge as a presentation like: \(\langle X_1,\dots X_n\mid X_3=4,X_2-X_1=2\rangle\), and morphisms cannot be concretely understood as functions on sets but rather show a tree of possible outcomes, like maybe you believe in Everett branches or whatever.)

In general if you postulate:

• … you live on some object in a category
• … time-evolution is governed by some automorphism \(H\)
• … you, the observer, have beliefs about your universe and keep forgetting some information (“coarse-grains the phase space”) — i.e. your subjective phase space is also an object in that category, which undergoes homomorphisms

Then the second law is just a tautology. The second law we all know and love comes from taking the universe to be a symplectic manifold, and time-evolution as governed by symplectomorphisms. And the point of Liouville’s theorem is really to clarify/physically motivate what the Jaynesian “uniform prior” should be. Here is some more stuff, from Yuxi Liu’s statistical mechanics article:

In almost all cases, we use the uniform prior over phase space. This is how Gibbs did it, and he didn’t really justify it other than saying that it just works, and suggesting it has something to do with Liouville’s theorem. Now with a century of hindsight, we know that it works because of quantum mechanics: We should use the uniform prior over phase space, because phase space volume has a natural unit of measurement: \(h^N\), where \(h\) is Planck’s constant, and \(2N\) is the dimension of phase space. As Planck’s constant is a universal constant, independent of where we are in phase space, we should weight all of the phase space equally, resulting in a uniform prior.