back
5 comments
The best study guide to Jaynes that I've found is from David Blower (sadly, recently deceased):

Information Processing: The Maximum Entropy Principle https://a.co/d/71tL5bw

He really takes apart the maximum entropy principle in a comprehensible way, to the point where one can see how to apply it to new problems.

(the volumes I and III are also good but not strictly necessary)

A simple example by Nassim Taleb:

https://www.fooledbyrandomness.com/blog/2021/09/07/estimatin...

Jaynes is brilliant, but you ought to take care when reading him. For example, AFAICT his digression on Gödel’s theorem in Logic of science is complete nonsense, and the rant against Kolmogorov-style axiomatization of infinite probability spaces in same isn’t much better.
“From many years of experience with its applications in hundreds of real problems, our views on the foundations of probability theory have evolved into something quite complex, which cannot be described in any such simplistic terms as ‘pro-this' or ‘anti-that'. For example, our system of probability could hardly be more different from that of Kolmogorov, in style, philosophy, and purpose. What we consider to be fully half of probability theory as it is needed in current applications – the principles for assigning probabilities by logical analysis of incomplete information – is not present at all in the Kolmogorov system.

“Yet, when all is said and done, we find ourselves, to our own surprise, in agreement with Kolmogorov and in disagreement with his critics, on nearly all technical issues. As noted in Appendix A, each of his axioms turns out to be, for all practical purposes, derivable from the Polya–Cox desiderata of rationality and consistency. In short, we regard our system of probability as not contradicting Kolmogorov's; but rather seeking a deeper logical foundation that permits its extension in the directions that are needed for modern applications.”

I've read in Quantum Information Theory papers, that Jaynes misunderstood Bell (he just didn't get it).

https://physics.stackexchange.com/questions/233203/has-jayne...

I just found this recent writeup on the subject, I didn’t have time to read it yet but looks interesting (he hasn’t been active on HN for almost two years, by the way).

https://scottlocklin.wordpress.com/2022/06/06/poking-holes-i...

Anybody who loves statistical mechanics has surely heard of Jaynes.

Unfortunately, those who like statistical mechanics seem few and far between. :(

Reminds me of a classic quote from a stat mech textbook:

“Ludwig Boltzmann, who spent much of his life studying statistical mechanics, died in 1906, by his own hand. Paul Ehrenfest, carrying on the work, died similarly in 1933. Now it is our turn to study statistical mechanics.”

David L. Goodstein, States of Matter

Well, who likes having it pointed out that all the air in the room huddling in a corner is a possibility, albeit very, very small ...
Well, I do... :-)

Less facetiously, I think Jaynes is becoming better known as Bayesian techniques have become more mainstream.

"rationalists" are obsessed with Jaynes, in as much as HN overlaps with that community I'd say a lot of people on here are familiar