Jaynes is brilliant, but you ought to take care when reading him. For example, AFAICT his digression on Gödel’s theorem in Logic of science is complete nonsense, and the rant against Kolmogorov-style axiomatization of infinite probability spaces in same isn’t much better.
“From many years of experience with its applications in hundreds of real problems, our views on the foundations of probability theory have evolved into something quite complex, which cannot be described in any such simplistic terms as ‘pro-this' or ‘anti-that'. For example, our system of probability could hardly be more different from that of Kolmogorov, in style, philosophy, and purpose. What we consider to be fully half of probability theory as it is needed in current applications – the principles for assigning probabilities by logical analysis of incomplete information – is not present at all in the Kolmogorov system.
“Yet, when all is said and done, we find ourselves, to our own surprise, in agreement with Kolmogorov and in disagreement with his critics, on nearly all technical issues. As noted in Appendix A, each of his axioms turns out to be, for all practical purposes, derivable from the Polya–Cox desiderata of rationality and consistency. In short, we regard our system of probability as not contradicting Kolmogorov's; but rather seeking a deeper logical foundation that permits its extension in the directions that are needed for modern applications.”
I just found this recent writeup on the subject, I didn’t have time to read it yet but looks interesting (he hasn’t been active on HN for almost two years, by the way).
Reminds me of a classic quote from a stat mech textbook:
“Ludwig Boltzmann, who spent much of his life studying statistical mechanics, died in 1906, by his own hand. Paul Ehrenfest, carrying on the work, died similarly in 1933. Now it is our turn to study statistical mechanics.”
It’s worth noting (much to my surprise) that the Feynman lectures on physics weren’t written entirely by Feynman. I’ve often wondered whether the wonderful conversational style I’ve associated with him is actually him, or one of his contemporaries.
Either way, this is a great chapter on probability. Thanks to whoever wrote it!
There’s an unpleasant point in the history of the Feynman lectures: them coming out as “The Feynman Lectures on Physics by Feyman, Leighton, and Sands” is a compromise solution for a dispute where Leighton and Sands wanted to be credited for editing the transcript into readable prose and Feynman considered that work to be purely mechanical and not worth any credit at all[1]. (There is apparently more uncredited work in there as well[2].) “Feynman didn’t understand editing is an art” is not much of a headline, but the compromise is still there, in huge letters right on the cover.
Thanks! Interestingly, the tape for Probability seems to be narrated by someone other than Feynman.
I should probably give some evidence to back up my claim that Feynman didn’t write all of the Lectures, but alas, it’s late. I think the credits for the rest of the authors were in the preface, or at the end. I just wish they’d gotten a little more credit.
“Early on, though, a small problem surfaced. Feynman had a long-time commitment to be absent from Caltech the third week of the fall semester, and so would miss two class lectures. The problem was easily solved: I would substitute for him on those days. However, to avoid breaking the continuity of his presentation, I would give the two lectures on subsidiary topics that, although useful to the students, would not be related to his main line of development.”
“The next stumbling block was more serious: choosing a title for the book. Visiting Feynman in his office one day to discuss the subject, I proposed that we adopt a simple name like “Physics” or “Physics One” and suggested that the authors be Feynman, Leighton, and Sands. He didn’t particularly like the suggested title and had a rather violent reaction to the proposed authors: “Why should your names be there? You were only doing the work of a stenographer!” I disagreed and pointed out that, without the efforts of Leighton and me, the lectures would never have come to be a book. The disagreement was not immediately resolved. I returned to the discussion some days later and we came up with a compromise: “The Feynman Lectures on Physics by Feynman, Leighton, and Sands.”
I think these people other than Feynman transcribed and edited the lectures into book form. This seems to have been the process with most (all?) books of his, "QED" and "The character of physical law" were also delivered as lectures and even "Surely You're Joking, Mr. Feynman!" was an interview originally that was later transcribed and edited.
Well, this is a fascinating rabbit hole. Apparently there’s some question whether Shakespeare himself was literate, since his parents and daughters seemingly weren’t.
Just keep in mind that the Shakespeare authorship conspiracy theory is the is the "the moon landing was fake" of the 1800s. The theory first gained popularly thanks to Delia Bacon in the 1850s, over 200 years after Shakespeare lived.
There's no evidence Shakespeare didn't write his plays and a lot of evidence he did (including multiple books and writings published during this life listing him as author or referring to him as an author).
Shakespeare true or not, I have often found that when one excels, others sit on the sidelines, stupefied in disbelief, then shout cries of delusion about the accolades before them.
Hence, Shakespeare cannot be real, for he excels you see...
I guess the GP is more about the idea that Shakespeare was just the first one to write all the folkloric ideas of his time in a format that people loved, instead of that unexplainable genius that created all those interesting stories. (Kinda like Disney. But we don't have the originals anymore.)
That one is a lot better accepted than the idea that he didn't write his works.
Somewhat related: "Shakespear's plays weren't written by him, but by someone alse of the same name" : an essay on intensionality and frame-based knowledge representation systems
by Douglas R. Hofstadter, Gray A. Clossman, Marsha J. Meredith.
I have a related question to this topic. Is probability axiomatic to reality? Does it exist on the same level as logic where logic is axiomatic to reality and just assumed to be true? Or is it simply a phenomenon arising from logic?
It seems like probability just happens to work without explanation? Intuitively this seems a bit strange since it feels as though probability should be derived from something else. Not sure if I'm correct here.
What confuses me even more is that I do know logic can be defined in terms of probability. Causal connections can be probabilistic. If A then 25% chance of B and so on.
Probability theory can be interpreted as an extension to logic where variables can take fractional values (rather than just be 0/false or 1/true).
E.g.,
((A or B) and C) = (A and C) or (B and C)
=> P[(A or B) and C] = P[(A and C) or (B and C)]
= P[(A and C)] + P[(B and C)] - P[(A and B) and (A and C)]
= P[(A and C)] + P[(B and C)] - P[A and B and C]
= P[A|C] P[C] + P[B|C] P[C] - P[A and B and C]
Notice the last couple lines -- this is the way in which probability extends logic. In you take the limit where P[A], P[B], P[C] = 0, 1, then the probability statement reduces to the logic statement at the top.
I already mentioned that logic can be defined in terms of probability above.
Here's the strange part. Let's say we make those probabilities to be 100% thus we have ordinary logic without probability.
Then let's create a physical closed Newtonian system: The interior of a cube with no gravity and a bunch of identical bouncing balls which obey Newtonian physics and therefore logic.
Let's say those balls all have a random initial velocity at a random direction but all those balls are initially positioned near one corner in the cube. Thus the balls from a position stand point start with low entropy. I use the term random, but it's not really random as you have perfect knowledge of these numbers, you know what they all are.
As time increases, entropy increases. The balls as a system begin to increase in entropy. Entropy continues to rise towards an equilibrium.
That is the question. Entropy is a probabilistic phenomenon. It occurs because ball configurations that are spread out have higher probabilities of occurring then balls that are concentrated in a corner. Thus given enough time the balls occupy the higher entropy state.
My question is, WHY does this occur. Probability (aka Entropy) is ARISING out of a perfect Newtonian system following perfect logic without any probabilistic extensions.
Can entropy be derived from Newtonian physics? The better question is, can probability be derived from Newtonian physics because entropy is in actuality a phenomenon of probability?
Any system that uses pure logic (and completely avoids probability in any part of its definition) exhibits this sort of rule as long as you introduce a sort of randomness into initial parameters above. Probability permeates everything.
> My question is, WHY does this occur. Probability (aka Entropy) is ARISING out of a perfect Newtonian system following perfect logic without any probabilistic extensions.
Probability can arise out of perfect information only when you throw away that perfect information and chose to describe the system probabilistically using imperfect information instead.
If you describe fully the state of a classical system - giving the positions and velocities of the particles in this case - there is no uncertainty at all. The volume in the 6N-dimensional phase space of the possible states collapse to a single point. The entropy is a function of the accessible volume in the phase space - it's minimised when the state is perfectly determined (it's defined to an additive constant and this minimum is conventionally set at 0).
If you describe the state only partially there is uncertainty in the description. There is a non-zero volume accessible in the phase space. The larger this volume the higher the entropy. Interestingly the volume - and thus the entropy - doesn't change in classical mechanics (Liouville's theorem).
In our perfect Newtonian system following perfect logic even if we don't know the precise state - and have a volume in phase space instead - we know how each of those potential states will evolve. There is no irreversibility, no changes in entropy. (Gibbs entropy is also conserved.) Only if we keep adding uncertainty / forgetting information as we go forward will entropy increase.
> You have not answered my question. You aren't getting it. Rising Entropy is just an example for illustrative purposes, entropy is simply a probabilistic phenomenon.
I thought that your question was how probability "appears" from classical mechanics and the answer is that you're putting it in - either by obligation (because you cannot describe the system perfectly) or by choice (because you decide not to do so even if you could).
Entropy arises when the classical state of the system is not identified precisely as a point in phase space and you have a volume instead corresponding to the possible physical states.
More uncertainty about the state, more volume, higher entropy. Less uncertainty about the state, less volume, lower entropy. No uncertainty about the state, no volume, zero entropy.
>> Only if we keep adding uncertainty / forgetting information as we go forward will entropy increase.
> Bro. Forgetting stuff doesn't change entropy. We already agreed that entropy is independent of knowledge.
We already agreed that the entropy depends on the description that we make of the system. By forgetting stuff or otherwise adding uncertainty you get a worse description than what you could have if you didn't - that means higher entropy. If the uncertainty doesn't increase somehow the entropy remains constant.
Entropy is a function of the volume in phase space and that volume is constant in classical -and quantum, for the matter- mechanics. If the equations describing the evolution are perfectly known the uncertainty doesn't have to increase.
It doesn't matter how much time passes, the particles cannot be _anywhere_: their possible positions in the future are determined by their possible positions in the past. But this constant volume of possible positions may get more and more spread around even though they used to be in a compact region.
By not making use of that additional information - possibly because we just can't afford that level of detail - we let uncertainty grow. That's what I meant by adding uncertainty / forgetting information. The volume in phase space representing the possible states is larger. The entropy of the degraded description of the system is higher.
You said that you where thinking about Gibbs entropy. He came up with the ideas above. Including coarse-graining - the “add uncertainty losing the details to allow for entropy to increase” part.
In Gibbs’ approach one represents the microstate of a physical system with N particles each with f degrees of freedom by a point X ∈ Γ where Γ, the phase space of the system, is a 2Nf-dimensional space spanned by the Nf momenta and Nf configuration axes. As the system evolves this representative point will trace out a trajectory in Γ which obeys Hamilton’s equations of motion.
Next, one considers a fictitious ensemble of individual systems (represented by a ‘cloud’, or a ‘fluid’, of points on phase space) each in a microstate compatible with a given macrostate (say, such and such energy in such and such pressure contained in such and such volume). The macroscopic parameters thus pick out a distribution of points in Γ. We then ascribe a normalized density function to the ensemble, ρ(p,q,t), and, except for entropy and temperature, the mean value of phase function with respect to ρ describes the system’s thermodynamic properties.[…]
All this is so very fine, but if Gibbs’ systems obey Hamilton’s equations of motion then the ‘cloud’ representing them in phase space swarms like an incompressible fluid. Consequently his ‘fine-grained’ entropy as defined in (7.9) is invariant under the Hamiltonian flow:
dSF/dt G(ρ) = 0. (7.10)
If there is a problem in SFG it is not just the fact that it does not move. Recall that TD entropy is defined only in equilibrium, so in order to construct a mechanical counterpart one only needs to find a function whose value at a later equilibrium state is higher than at an earlier equilibrium state.21 But since the macroscopic parameters change between the two equilibrium states, the Gibbs’ approach has no problem in doing this just by defining a new ensemble with a new probability distribution for the new equilibrium state and this will match the thermodynamic entropy as before.
[…] it is not fair to use the macro-parameters, which are supposed to be derived from the micro-parameters, in order to construct the latter. In other words, the ensemble at later equilibrium state should be the Hamiltonian-time-evolved ensemble of the earlier equilibrium state, otherwise the system is not governed by Hamilton’s equations as one originally presupposes. Thus, if one wants to use Gibbs’ fine-grained entropy as a mechanical counterpart to TD entropy, then one must abandon standard, Hamiltonian, dynamics since it does not connect the two fine-grained equilibrium states.
That this is the true problem with Gibbs’ fine-grained entropy escaped many commentators, and as a result the foundations of SM were soon piled with a lot of dead wood. Stemming from the famous Ehrenfests’ paper (1912, 43–79) where Boltzmann’s students complained on Gibbs’ treatment of irreversibility by categorizing it bluntly as “incorrect”, the last century was consumed with attempts to find a monotonically increasing function as a counterpart for TD entropy.
One way to achieve this goal is to follow Gibbs himself, who introduces the mathematical trick of ‘coarse graining’ and devises new notions of entropy and equilibrium. In this approach one divides Γ into many small finite cells of volume ω and then takes the average of ρ over these cells.
The statistical version. The equation is independent of knowledge.
Your logic is flawed. Think about it. Knowing more or less about a system does not change it's entropy.
If I knew the position and velocity of every particle in a gas chamber does that somehow lower it's temperature? No.
Macro-states summarize perfect information. But with knowledge of perfect information, you still have the ability to summarize it. Or in other words, probability still exists even when you have perfect knowledge of everything and the future.
If you mean Boltzmann's formula it will be valid for systems in thermodynamical equilibrium. Is your system in thermodynamical equilibrium?
You talk about macrostates. How do you define a macrostate for your initial configuration? Why this one and not another? Does it change over time?
Think of it from another angle. You have a box with some of these balls inside - all you know is that they all have the same energy. Do you agree that the macrostate won't change over time and the entropy will remain constant?
However, if you knew the positions they will be concentrated in some regions more than in others. That's what you called lower entropy in your example.
So what is it, does knowing more about a system change it's entropy or not?
> If you mean Boltzmann's formula it will be valid for systems in thermodynamical equilibrium. Is your system in thermodynamical equilibrium?
Gibbs
>You talk about macrostates. How do you define a macrostate for your initial configuration? Why this one and not another? Does it change over time?
I'll choose something arbitrary. Temperature as measured by a thermometer is the macrostate. Every possible configuration of particles (microstates) that causes mercury to rise to a certain degree represents a different macrostate. And yes it changes with time; even at equilibrium.
>Think of it from another angle. You have a box with some of these balls inside - all you know is that they all have the same energy. Do you agree that the macrostate won't change over time and the entropy will remain constant?
No don't agree. If balls are located on one side of the box and the thermometer is on the other side of the box. The thermometer reads nothing. When the balls increase in entropy they collide with the thermometer producing a reading.
>However, if you knew the positions they will be concentrated in some regions more than in others. That's what you called lower entropy in your example.
Yeah I already don't agree with you, so the rest of your argument is gone. If you define macrostate as total energy in a system then yeah it never changes. But that's not the definition of macrostate. It's just one arbitrary choice you have chosen.
> If you define macrostate as total energy in a system then yeah it never changes. But that's not the definition of macrostate.
The more I think about your reply the less sense it makes to me.
What do you think is the definition of macrostate?
The standard notion in statistical mechanics is that if we have, for example, a volume of gas in equilibrium at (constant) ambient temperature and we measure the pressure it doesn’t change. The macrostate doesn’t change. That’s what being in equilibrium means. The macrostate is in that case defined by the variables P,T,V. If all you knew was the value of these three variables and they didn’t change how could the macrostate - or the entropy - change?
You tell me that if you know the precise position of the particles of that gas then the macrostate and the entropy change all the time.
But you also tell me that “Knowing more or less about a system does not change it's entropy.” Which is in flagrant contradiction with the two previous paragraphs.
>The more I think about your reply the less sense it makes to me.
I think the issue is more with your understanding then my explanation.
>What do you think is the definition of macrostate?
We first introduce the very fundamental statistical ideas of microstates and macrostates.
Given a system (e.g., a gas), we view it as built from some elementary constituents, (e.g.,
molecules). Each constituent has a set of possible states it can be in. The thermodynamic
state of the system (which characterizes the values of macroscopic observables such as
energy, pressure, volume, etc. ) corresponds to many possible states of the constituents
(the molecules). The collection of states of all the constituents is the microstate. To keep
things clear, we refer to the macroscopic, thermodynamic state as the macrostate. The
vast disparity between the number of possible macrostates versus microstates is at the
heart of thermodynamic behavior! The number of distinct microstates giving the same
macrostate is called the multiplicity of the macrostate. The multiplicity is a sort of micro-scopic observable which can be assigned to a macrostate.
>The standard notion in statistical mechanics is that if we have, for example, a volume of gas in equilibrium at (constant) ambient temperature and we measure the pressure it doesn’t change. The macrostate doesn’t change. That’s what being in equilibrium means. The macrostate is in that case defined by the variables P,T,V. If all you knew was the value of these three variables and they didn’t change how could the macrostate - or the entropy - change?
This notion is wrong. It CAN change. It just has an extreme low probability of changing from equilibrium to some low entropy state. The probability is low enough that you practically don't need to consider it, but you must consider it from a technical standpoint.
If by sheer luck all gas particles moved to the exact left side of the container. My measurement tool (thermometer) for macrostate was on the right side of the container then it registers zero. There is nothing in the laws of physics that prevents this from happening. Only probability makes this situation unlikely to happen.
>You tell me that if you know the precise position of the particles of that gas then the macrostate and the entropy change all the time.
No. I'm saying that the temperature reading on the thermometer a macroscopic measurement is INDEPENDENT of knowledge. Your mind doesn't control microstates and thus the macrostate of the system.
>But you also tell me that “Knowing more or less about a system does not change it's entropy.” Which is in flagrant contradiction with the two previous paragraphs.
Flagrant? You're offended? Well you can leave if you're offended. Using words like this also Offends me so we can end the conversation.
> I think the issue is more with your understanding then my explanation.
That's what I said, that it doesn't make sense to me... hopefully it does make some sense to you!
> This notion is wrong. It CAN change. It just has an extreme low probability of changing from equilibrium to some low entropy state.
You also said before that the macrostate "changes with time; even at equilibrium".
There is no point in discussing fluctuations if we can't agree on the notion of equilibrium and the corresponding macrostate defined by its (average) macroscopic properties.
It's fine that you have your very own notion of things. Unfortunately it makes the discussion difficult.
>There is no point in discussing fluctuations if we can't agree on the notion of equilibrium and the corresponding macrostate defined by its (average) macroscopic properties.
You mean microscopic. Equilibrium would then be the temperature that the thermometer is most likely to read at any point in time given infinite time. Do you agree or disagree. If you agree it's fine.
So with time eventually the particles will configure itself into a macrostate that reads 0 on the thermometer.
“An isolated system is in equilibrium if the sharp constraints and expectation values which define its macrostate pertain to constants of the motion only.”
“Whenever the macroscopic data include an expectation value, a measurement of the pertinent observable may yield a wide range of outcomes.”
I’m not making up the idea of an equilibrium state being associated with a macrostate.
"The equilibrium macrostate is that with the most microstates, and this is the state of greatest entropy. A macroscopic flux is most likely in the direction of increasing entropy."
>I’m not making up the idea of an equilibrium state being associated with a macrostate.
I know you aren't. TBH not even sure why equilibrium needs to be brought up. The topic "is entropy is independent of knowledge?" You disagree with that statement.
Macrostate is defined in terms of microstates and is independent of your knowledge of the exact configuration of the microstates. You have some blunt tool like the thermometer that gives you macroscopic data and that hides the microstates from you.
But if I had some precision tool that reads the position of every particle I can still identify how that configuration of microstates will influence the blunt tool. The blunt tool hides knowledge, but your knowledge of the microstate does not change the reading on that blunt tool.
Because entropy is defined in termms of macrostate it stays the same regardless of which tool you used to do the measurement.
See? We agree that there is an equilibrium macrostate. When I fist asked you to clarify what did you mean by macrostate you told me that “it changes with time; even at equilibrium.”
We agree that the entropy depends on the macrostate. Note that the macrostate is our description of the system and depends on how we choose to describe it which normally depends on what are the constraints, how it was prepared., etc. It’s not just a property of the position of those balls.
It’s because we agree that Gibbs’ entropy is a function of the macrostate that I asked how did you define it in your example. You told me: “Temperature as measured by a thermometer is the macrostate. Every possible configuration of particles (microstates) that causes mercury to rise to a certain degree represents a different macrostate.”
I asked “How does your configuration where the balls were near one corner in the cube cause mercury to rise to a different level than the configuration where they occupy a larger volume near the center?” and the answer “The balls have to touch thermometer” doesn’t cut it. The balls don’t touch the thermometer in either case.
You seemed to imply that the higher concentration means a different macrostate with lower entropy. Or maybe the low entropy in you example is because the balls are near a corner?
Anyway, it would indeed have been easier to say that definition of macrostate included the density of particles in each octant of the cube - or something like that.
>See? We agree that there is an equilibrium macrostate. When I fist asked you to clarify what did you mean by macrostate you told me that “it changes with time; even at equilibrium.”
I still stand by my statement. Even at equilibrium it can lower in entropy. The equilibrium is simply the highest entropy state.
>I asked “How does your configuration where the balls were near one corner in the cube cause mercury to rise to a different level than the configuration where they occupy a larger volume near the center?” and the answer “The balls have to touch thermometer” doesn’t cut it. The balls don’t touch the thermometer in either case.
I stated this is pedantism. The concept and intuition remain true. I changed the definition so that it's a volume around the thermometer if the particle is in that volume and heading for the thermometer is counts as a collision.
I stated all of this already.
>You seemed to imply that the higher concentration means a different macrostate with lower entropy. Or maybe the low entropy in you example is because the balls are near a corner?
Yes. The higher concentration has a lower probability of occurring. And occupies a different temperature reading on the thermometer. Each temperature reading is a different macrostate.
>Anyway, it would indeed have been easier to say that definition of macrostate included the density of particles in each octant of the cube - or something like that.
Sure, Divide the box into a bunch of cubes. If 1 or more particles are in the cube then that cube represents 1, otherwise 0. Add those numbers up and that represents a macrostate.
The inuition remains the same. For all particles to be concentrated in 1 cube is a very low probability. And the macrostate will be quite low too. With enough cubes and boxes such a state has a very low probability of occuring.
But all of this is, again, independent of your knowledge of where the particles are in each cube.
> And occupies a different temperature reading on the thermometer.
You are unable to explain how reducing the space occupied by the initial configuration you proposed could change the temperature - which you say is 0 in all those cases - or the macrostate - which you call absolute zero macrostate in all those cases. The entropy would be the same whenever the particules are more concentrated than in your example - even though they would have lower probability of ocurrence. Don't you agree?
> With enough cubes and boxes such a state has a very low probability of occuring.
Sure. The thing is that if you calculate an entropy using Gibbs formula from the distribution of microstates for a given macrostate the value that you obtain depends on how many cubes are used to define the macrostate. There is no entropy of the microstate - the entropy depends on how you decide to define the macrostate. If the lattice is fine enough, and the particles indistinguishable, in the limit the entropy is zero - the macrostate becomes the same as the microstate and one single microstate is possible.
It all depends on the description we make of the system. The "I have balls in a box and I know their positions" example was incomplete.
If for example I have N of those balls in equilibrium in an isolated box of volume V and I know the energy E I know the equilibrium macrostate. The energy won't change because it's isolated. The macrostate will not change. The (equi)probability distribution of the possible microstates corresponding to the macrostate won't change. Gibbs entropy which is calculated using that distribution won't change.
If by sheer luck all gas particles moved to the exact left side of the container the energy wouldn't change, the macrostate wouldn't change, the entropy wouldn't change - it doesn't matter how unlikely that is. It doesn't matter if you know the position of each particle. The entropy for the thermodynamical system described in the previous paragraph is independent of your knowledge of where the particles are or how unlikely you think their positions are.
I agree that you could use alternative ways of defining the macrostate where that happens! (And calculating Gibbs entropy will give different results. One can even get zero entropy using the microstate as macrostate - and one may actually want to do that if the microstate is known!)
> Even at equilibrium it can lower in entropy. The equilibrium is simply the highest entropy state.
I would understand that you said either:
"the equilibrium is the highest entropy state but there may be fluctuations that shift the system temporarily out equilibrium"
or
"the equilibrium is the highest entropy state and possibly a distribution of states around it."
Both concepts are used. When you say that the equilibrium is simply the highest entropy state but the equilibrium entropy can also be lower I'm not sure if it can be read as one of those or you're saying something else entirely.
>You are unable to explain how reducing the space occupied by the initial configuration you proposed could change the temperature - which you say is 0 in all those cases - or the macrostate - which you call absolute zero macrostate in all those cases. The entropy would be the same whenever the particules are more concentrated than in your example - even though they would have lower probability of ocurrence. Don't you agree?
You're unable to read my explanation which i've already repeated twice. Go back and look it up. I even went with your cubical definition.
I don't agree. Entropy is lower when the current macrostate has a low probability of occuring.
>There is no entropy of the microstate - the entropy depends on how you decide to define the macrostate
The macrostate is defined in terms of possible microstates. Thus Entropy relies on both microstate and macrostate.
>If by sheer luck all gas particles moved to the exact left side of the container the energy wouldn't change, the macrostate wouldn't change, the entropy wouldn't change - it doesn't matter how unlikely that is.
Wrong. Energy wouldn't change. Macrostate changes. Entropy changes.
>It doesn't matter if you know the position of each particle. The entropy for the thermodynamical system described in the previous paragraph is independent of your knowledge of where the particles are or how unlikely you think their positions are.
Isn't this my point? And weren't you against my point? Now you're in agreement? My point was entropy is independent of knowledge. Your point was that it is dependent.
>I agree that you could use alternative ways of defining the macrostate where that happens! (And calculating Gibbs entropy will give different results. One can even get zero entropy using the microstate as macrostate - and one may actually want to do that if the microstate is known!)
Except you don't have to do this. My point remains true given MY stated definitions of macrostate.
>Both concepts are used. When you say that the equilibrium is simply the highest entropy state but the equilibrium entropy can also be lower I'm not sure if it can be read as one of those or you're saying something else entirely.
I made a true statement that from what I can gather you agree it's true. You're just trying to extrapolate my reasoning behind the statement. It's pointless. Example: If I give you a number, say -1, do I mean i^2 or 0 - 1? because one of those equations is impossible to express in reality.
>>If by sheer luck all gas particles moved to the exact left side of the container the energy wouldn't change, the macrostate wouldn't change, the entropy wouldn't change - it doesn't matter how unlikely that is.
> Wrong. Energy wouldn't change. Macrostate changes. Entropy changes.
That's my thermodynamical system - that I decided to describe using the state variables N, V, E. S_kgwgk = constant
I have agreed that you can chose to define macrostates as you wish! (You cannot measure them though. You don't have access to my system. They are purely hypothetical.) And then you can say things like "If the position of the particles in your system is X, S_deltasevennine = whatever."
Hopefully you will agree that someone else could choose to define the macrostate differently and say things like "If the position of the particles in kgwgk's system is X, S_anon = somethingelse."
Let's imagine that the position of the particles is actually X. What is the entropy of the system then? S_kgwgk? S_deltasevennine? S_anon? They are all different!
If you think that you can define macrostates for my system in some arbitrary way (your own words: "I'll choose something arbitrary.") and others can't - or that yours are somehow more real - I really wonder why.
And as I said, someone could also come and say "If the position [and velocities] of the particles in kgwgk's system is X, S_vonneumann = 0." [a complete definition of the microstate requires knowing both position and momentum - there might have been some ambiguity about that before but it shouldn't distract us from the main points]
I thought we agreed on my definition. If you want to make your own definitions of macrostate sure. No rule against that, I just don't see your point.
>If you think that you can define macrostates for my system in some arbitrary way (your own words: "I'll choose something arbitrary.") and others can't - or that yours are somehow more real - I really wonder why.
I really wonder why you even say this. Anyone can make up a macrostate we just choose to agree on one for discussions sake. But in the real world things like pressure and temperature are some universal agreed upon ones. I simply made one up so we can center our discussion around it and you took it into other territories.
You can switch up definitions all you want. But no matter your definition of macrostate, this remains true:
Entropy is independent of knowledge. Your initial argument was against that. I haven't seen you make any argument in your own favor. Just side discussions on who's definition of macro state to use.
> If you want to make your own definitions of macrostate sure.
Note that I was the first one to propose a particular way to define the macrostate of the system - using energy - trying to understand what you meant. Your reply was "that's not the definition of macrostate. It's just one arbitrary choice you have chosen." And you gave your own arbitrary choice "I'll choose something arbitrary. Temperature as measured by a thermometer".
The point is that they are all arbitrary. (Energy is a conserved quantity for a closed system so it's arguably more natural - but let's say that any choice is equally arbitrary.)
> But no matter your definition of macrostate, this remains true:
> Entropy is independent of knowledge.
Maybe be can agree that [Gibbs] entropy [which is a property of the distribution of microstates corresponding to our description of the system based one some macroscopic properties] is independent of knowledge [other than about that particular macroscopic description].
The discussion started with me trying to understand what did you mean by "low entropy" when you said:
"Let's say those balls all have a random initial velocity at a random direction but all those balls are initially positioned near one corner in the cube. Thus the balls from a position stand point start with low entropy."
We agree that if the box is isolated and we define the state using N,V,E there is one single macrostate and one single value for the entropy - independent of the position of the particles. We agree that we can define thermodynamical systems using other variables and calculate other entropies. We agree that the entropy is not determined by the configuration of the particles alone.
(Even when a thermodynamical system is defined the entropy is not necesarily a function of the microstate when it refers to only a part of the system. The microstate may not fully determine the macroscopic description. If that box is in equilibrium in a heat bath the temperature is fixed and there is some corresponding entropy. But the same microstate can happen for equilibrium systems at different temperatures and therefore with different entropies.)
You said that you're using temperature as measured by a thermometer as the macroscopic property describing the system. And I'm still trying to understand how "higher concentration" means "lower entropy" even in the context of your own arbitrary choice.
Temperature works well as a state variable for a system in thermal equilibrium which has the same temperature everywhere. You mention the level of mercury and the way a thermometer works is by reaching thermal equilibrium between the mercury and the thing being measured. Note that if for some reason all the particles in the gas go away from the thermometer for a second the temperature of the mercury won't change (ignoring that it will radiate energy over time). The reading of the thermometer doesn't drop to zero just because there is nothing there.
Let's assume that you are somehow measuring the local temperature in a small region around some specific point. What you measure will not depend in any way on the particles that are elsewhere. The particles in the rest of the box could be well spread or all near one corner and you would have no reason to say that the latter configuration is lower entropy than the former based on your macrostate.
Another way to look at it: you reasoning seems to be that the state with the particles near one corner has low entropy because they are close to each other and there is some temperature reading in your far-away thermometer lower than the equilibrium temperature and then the entropy is low. If a similar cluster of particles was close to the thermometer rather than far from it I guess that you will tell me that the temperature reading would be higher than in equilibrium (and the entropy would also be lower).
But if you put that cluster of particles at some intermediate position the reading of the thermometer will be the same as in equilibrium! Your macrostate will be the same as in equilibrium. Your entropy will not be lower than in equilibrium. Even though you said -if I understood you correctly- that the entropy should be lower because the higher concentration has a lower probability of occurring.
> Note that I was the first one to propose a particular way to define the macrostate (energy) of the system to understand what you meant. Your reply was "that's not the definition of macrostate. It's just one arbitrary choice you have chosen." And you gave your own arbitrary choice "I'll choose something arbitrary. Temperature as measured by a thermometer".
No. You explicitly ASKED for my definition, then I said that.
When I said that's not the "definition of macrostate" I did not make an arbitrary choice there. I simply stated that IF you think energy was the definition of macrostate, then you are wrong.
> Maybe be can agree that [Gibbs] entropy [which is a property of the distribution of microstates corresponding to our description of the system based one some macroscopic properties] is independent of knowledge [other than about that particular macroscopic description].
This is the point of the ENTIRE thread starting from your INITIAL reply. Technically this argument is over. You weren't in agreement with me, now you are, so you were wrong and I was right. This sentence admits that.
The rest of the stuff here is side tangents and it's hard to see the main point. I can entertain it though for a little bit but this here is essentially the end.
>The discussion started with me trying to understand what did you mean by "low entropy" when you said:
This discussion started with you saying that if I know the microstate of the system entropy is zero. If you wanted to understand What I thought about "low entropy" then this was NEVER stated. The thread is based off what is stated and what is not stated. It is not based off of what your internal thoughts and intentions are. If you want the topic to based off your own thoughts, they need to be expressed explicitly in statements. You did so just now, but way too late.
>Note that if for some reason all the particles in the gas go away from the thermometer for a second the temperature of the mercury won't change (ignoring that it will radiate energy over time). The reading of the thermometer doesn't drop to zero just because there is nothing there.
This is pedantic. Obviously. I can take it into account. Let's say the greatest average speed measured of a particle and the time it takes for a particle at this speed to travel across the box is the time it takes for the thermometer to drop to zero degrees from the maximum temperature. It's quite fast but particles within the vicinity will keep the mercury level stable but if they were concentrated in a corner, the mercury drops fast enough to change the reading of the thermometer.
>Let's assume that you are somehow measuring the temperature in a small region somewhere with a very high reaction time. What you measure will not depend in any way on the particles that are elsewhere. The particles in the rest of the box could be well spread or all near one corner and you would have no reason to say that the latter configuration is lower entropy than the former based on your macrostate.
I specifically defined it as a thermometer to make location matter. Switching the location of the thermometer is switching the definition as well.
I really don't see what your point is. You think I'm wrong about something? What am I wrong about?
>But at some intermediate point the reading of the thermometer will be the same as in equilibrium. Your macrostate will be the same as in equilibrium. Your entropy will not be lower than in equilibrium. Even though you said -if I understood you correctly- that the entropy should be lower because the higher concentration has a lower probability of occurring.
Your point seems to be buried in here somewhere and I can't parse it. There is a macrostate that is equilibrium, yes.
Are you referring to mercury level mid transition? This is pedant-ism to the max if you are. Yes the mercury level will display the WRONG temperature if it's mid transition. Not willing to constantly adjust the model to little flaws you find. Last time: Let's switch to a digital thermometer that displays temperature at time intervals that are equal to the length of time it takes for the maximum speed particle to travel across the box. There is no transition value now. All temperature readings reflect an instantaneous observed truth at an instantaneous point in time, but that temperature is displayed at non-instantaneous intervals.
It also seems to me that your definition of macrostate is meaningless. The total energy of the universe is hypothetically the same all the time. If the macrostate was just energy there's no point to it, because entropy would then be an unchanging constant.
I think we're done here. I'm just trying to guess what you're driving at. You'll need to clarify your point if you want me to continue. What exactly are you trying to say here?
Ok. If you have a system at constant energy they are the same (all the microstates have the same probability). If you have a system a constant temperature a microstate can always correspond to different macrostates (it seems you’re not concerned about that ambiguity though).
In any case, the entropy is a property of (the ensemble of microstates that conform) the macrostate and not a property of the microstate. For a given macrostate you cannot use Gibbs entropy to talk about low-entropy and high-entropy microstates.
> Temperature as measured by a thermometer is the macrostate.
Every possible configuration of particles (microstates) that causes mercury to rise to a certain degree represents a different macrostate.
How does your configuration where the balls where near one corner in the cube causes mercury to rise to a different level than the configuration where they occupy a larger volume near the center?
> And yes it changes with time; even at equilibrium.
What’s your definition of equilibrium? It’s hard for me to see what are we talking about, really.
> If balls are located on one side of the box and the thermometer is on the other side of the box. The thermometer reads nothing.
One could also say that if the balls are located anywhere within the box the thermometer reads nothing. The thermometer reading exists only when enough time has passed and the mercury-balls composite system is in equilibrium.
> When the balls increase in entropy they collide with the thermometer producing a reading.
Apparently you say that the entropy increases because the macrostate changes. But the macrostate is undefined until the balls hit the thermometer. But the entropy has to increase for the balls to move. It’s all a bit confusing, don’t you think?
Won’t the balls collide with the thermometer even if the entropy doesn’t change? It’s their movement that will take them there.
Consider your state - or any other state, for that matter - at t=0 and the state at t=1ps where the balls have advanced slightly. Is the macrostate different? How? Is the entropy different? How?
>How does your configuration where the balls where near one corner in the cube causes mercury to rise to a different level than the configuration where they occupy a larger volume near the center?
The balls have to touch thermometer. Otherwise the thermometer observes nothing. Deriving the exact formula of this interaction is complex but the intuition makes sense.
Interaction with the thermometer at a certain energy level produces a macrostate at Temperature T that is high probability. You see this macrostate throughout your lifetime. But nothing precludes the low probability macrostate (where all balls are at another corner of the box) from occuring. It's just that microstate has such a low probability of occuring you never see it. When nothing touches the thermometer, that temperature reading is the absolute zero macrostate.
>Apparently you say that the entropy increases because the macrostate changes. But the macrostate is undefined until the balls hit the thermometer. But the entropy has to increase for the balls to move. It’s all a bit confusing, don’t you think?
No. The macrostate is never undefined. I defined it as the measurement on that thermometer. Whatever you read on that thermometer (let's assume mercury levels move instantaneously and not be pedantic) then THAT is the current macrostate. I chose this set of macrostates, thus I pick that definition. There is no notion of undefined, the thermometer always HAS a reading.
>Won’t the balls collide with the thermometer even if the entropy doesn’t change? It’s their movement that will take them there.
They can't collide with the thermometer if they're on the other side of the box.
>Consider your state - or any other state, for that matter - at t=0 and the state at t=1ps where the balls have advanced slightly. Is the macrostate different? How? Is the entropy different? How?
Macrostate doesn't change if they're still far away from the thermometer. Still absolute zero. Only the microstate changed. The absolute zero macrostate includes all configurations of particles that will cause the thermometer to read 0. The probability of this macrostate occuring is quite low.
Entropy in this state is low. It increases as more and more particles interact with the thermometer.
The most probable series of events is that particles on the left corner of the box will begin to spread more evenly around the box. More and more particles will begin interacting with the thermometer causing the temperature to rise until it reaches some equilibrium. What can occur is all particles can by sheer luck suddenly concentrate to another corner of the box, but this is a low probability event.
To bring it back around to the main point. All of this is independent of your knowledge of the microstate. Your knowledge does not effect the outcome.
> The absolute zero macrostate includes all configurations of particles that will cause the thermometer to read 0. The probability of this macrostate occuring is quite low.
"Let's say those balls all have a random initial velocity at a random direction but all those balls are initially positioned near one corner in the cube. Thus the balls from a position stand point start with low entropy."
In every single initial configuration the particles will be at some distance from the thermometer and will cause it to read 0 according to you reasoning (until some time passes). I guess they all correspond to the same "absolute zero macrostate".
What I still don't see is how do you calculate the entropy using Gibbs entropy formula in such a way that the entropy is lower or higher for some initial configuration depending on the position of the particles. Gibbs entropy is not a property of the microstate - it's a property of the macrostate.
Probability, seems to me, cannot be fundamental, because, a machine built to flip a coin with always the same pressure, can be adjusted to always give heads, or tails. In such a setup there won't be probability.
From George Boole's The Laws of Thought, p.244: "Probability is expectation founded upon partial knowledge. A perfect acquaintance with _all_ the circumstances affecting the occurence of an event would change expectation into certainty, and leave neither room nor demand for a theory of probabilities."
Can we deduce from this that nature is not probabilistic?
It is true that (at least above the level of quantum physics) we tend to believe that reality is deterministic. When the meteorologist gives a chance of rain, in truth if they had perfect forward information on all of the clouds and pressure systems, they would simply declare whether or not there was future rain. In cases like physical phenomenon, you can think of observed uncertainty or chance as being a product of the exact settings of the unmodeled but deterministic factors underlying a particular outcome, or else errors in the functional form of the model with respect to the measure. We tend to assume that unmodeled factors are as-if orthogonal to the causes we are interested in modelling, and thus zero-centered, and our models minimize error predicated on this assumption.
In your proposed flipping model, there are likely to be very small physical imprecisions (vibrations in the flipper, say, drifting tension of some kind of spring or actuator, or small amounts of circulating air, or perhaps tiny imprecisions in the way the coin is loaded into a slot). The machine might always flip heads, but it's still possible to say that whatever arbitrary degree of certainty you need to model the coin's behaviour in the air to achieve 100% accuracy, there could still be arbitrarily smaller error below that threshold, and we'd view this as "randomness" even if it isn't by the laws of physics.
Not a physicist, but your argument is pretty unconvincing in that it relies entirely on intuition about classical physics, ignoring quantum phenomena entirely. If one were to argue that probability were fundamental, they'd very likely start by describing wave functions, which are probabilistic and seem pretty close to fundamental to observable reality.
There are two things we mean by "probability". The first is propensity, and this has clear links to information theory, as another person commented about already.
It's important to emphasise that in terms of propensity, it doesn't matter whether or not the event has occurred, what matters is your knowledge about it. A flipped fair coin has a definite side up (as can be verified by a silent third observer) but for you, who has not yet observed which side it is, your best guess is still either side with 50 % probability.
Similarly, if you only know there's a soccer game going on, you might guess that the stronger team will win with 60 % probability (based on historic frequencies of exchangeable situations), but someone who has seen the score and knows the weaker team has a lead and knows there's only a few minutes left of the game will judge there to be a 2 % probability the stronger team wins. Same situation, different information, different judgements.
That's the first meaning of "probability". What we also mean with that word is "the rules of probabilistic calculation". These are based on mathematical ideas like coherency (if one of two things can happen, their probabilities should add up to 100 %) and can definitely be taken as axiomatic.
All of this is not an answer to your question, but it might make the discussion richer.
There are multiple interpretations (corresponding to your first part). One of them is indeed about “propensities” but the most common ones are about “frequencies” and about “uncertainty”.
For you it isn't 1/6 but for him it is. From your perspective can you derive from the knowledge you have why that for him the probability is 1/6?
And I mean derive as in derive the mathematical probability from all your knowledge about physics and the dice to explain why the lack of knowledge appears to him as a probability.
This is the essence of my question. We assume it. Probability is taken to be true for no reason, similar to how we just assume logic to be true.
Probability is an abstract concept so if he has a really good model of the universe there is no reason he couldn’t have better information.
We don’t really know if “god plays dice” so to speak. We probably will never fully understand the underlying fabric of the universe even with lots of theories and experiments.
It is like we get to run valgrind on the program but never see the source code.
> Probability is an abstract concept so if he has a really good model of the universe there is no reason he couldn’t have better information.
It's not about reasoning. Sure he could have better information but this isn't the point. It's all about WHY probability works. Why are these laws suddenly obeyed when there's a lack of information?
Do we assume it's fundamental? Or can the laws of probability be derived from deeper axioms, like logic.
Perhaps you're just kidding around, but of course that's not good enough for a definition.
It doesn't handle continuous random quantities. It doesn't even handle situations with a discrete outcome but where counting the possible cases isn't well-defined (Buffon's needle being one, but an even better one being the chance of a tossed thumbtack landing point-up). It also doesn't handle cases where symmetry or physics can give the answer, but you can't count cases because they aren't finite or aren't necessarily a-priori equiprobable.
Probability does not exist in reality as it's open ended.
It does however exist in ex a deck of cards and a game that's defined.
Probability is often being misused to say things about reality though. You see that especially in computer simulation whether used in economics, weather etc.
Different initial conditions are put into the models and simulated. And the probability is calculated based on what the majority of those models say.
But those initial conditions are guesses not actual objective explanations. If they were you only needed to run one simulation rather than a range.
A lot of statistics is pure placebo. Purely retrospective.
In reality it either is or it isn't. If you have good explanations like we do in physics you don't need probability.
David Deutsch IMO has the most sane rebuttal of the probability.
> In reality it either is or it isn't. If you have good explanations like we do *in physics you don't need probability*.
So... are you saying Statistical Mechanics (to give an example) is not part of Physics?
In real life, the amount of information you have (and can have) about a physical process is limited. You can either throw your hands up and say "we can't know for sure", or you can use probabilities to try to get somewhere.
How do you define the position of an electron without using probabilities?
Just because it's limited doesn't mean I can assign probability to it.
As I said. The difference is closed and open systems.
No problem with probability of getting a given card based on what have already been dealt in ex a game.
The problem arises when it's applied to predicting reality in open ended systems as I said weather, economics, climate etc. and where history is being used as some sort of benchmark of the future.
There is no probability whether an astroid is on the path towards earth. It either is or it isn't.
There is a probability, it can be described based on what cards have already been dealt. It does not change whether that specific card is an ace or not.
Do we also agree that - even if you don’t - other people are able to conceive a probability that the next card in the deck is an ace, a probability that I was born on a Saturday, a probability that Germany wins the World Cup, etc.?
If you can pick Brazil or Morocco to win $1000 in case the one you choose wins the World Cup which one do you pick? Why?
Are you really indifferent between them because you cannot conceive how saying that one is more likely to win than the other could have any relationship to reality?
There is nothing in the mathematical definition of probability that prevents you from applying the formalism to “guessing”.
The axiomatic definition of probability is about consistency - not about explanations.
Whatever your definition of probability is it seems too restrictive.
You may have another name for the “this is more likely than that” statements and the related reasoning processes essential in science, technology and survival in the world.
Many people call that “probability” and sucessfully apply probability theory to reason about those things.
You’re missing the bigger point that both 50/50 and 80/20 are valid probability assignments as far as the mathematical theory of probability is concerned.
If we were talking about a deck of cards there would be no causal connection between the probability you assign to the next card in the deck and the next card in the deck.
Note that for the deck of cards different people can also have different probability assignments! Maybe you saw the first 26 cards as they were dealt face up, I saw them plus the card at the bottom of the deck, someone else has just arrived and has seen only the latest card dealt. Each of us will calculate different probabilities for the next card but all of them are unrelated to reality where the next card in the deck is fixed but unknown to us. Can any of us “explain the outcome”?
Mathematical theory has no causal effect on reality either. So no I am not missing some bigger point. I might as well make a metaphysical claim like. Thor will make x happen.
Ok, so probability doesn't exist at all for you [1] and the deck of cards discussion is just a red herring.
If your argument is that probability can't exist because if can't have a causal effect on reality maybe you should try to define and use probability as it's usually done - with the understanding that 'having a causal effect on reality' has nothing to do with it.
Probability can be used to say things about reality. 'Things' like 'what we know or expect'. Probability is about uncertainty. Statistical mechanics will say things like 'the average energy in that system is whatever' - this is something about reality. Particle physics will say 'the half-life of that muon is whatever'. That's physics as far as my defition of physics goes.
Can physics - in your view - explain that if you put together a glass of hot water and a glass of cold ethanol you end up with a warm mixture of water and ethanol? That "causality" only exists in a probabilistic sense after all...
——-
[1] You are in good company. Bruno de Finetti wrote:
“My thesis, paradoxically, and a little provocatively, but nonetheless genuinely, is simply this:
PROBABILITY DOES NOT EXIST
The abandonment of superstitious beliefs about the existence of the Phlogiston, the Cosmic Ether, Absolute Space and Time, . . . or Fairies and Witches was an essential step along the road to scientific thinking. Probability, too, if regarded as something endowed with some kind of objective existence, is no less a mis- leading misconception, an illusory attempt to exteriorize or materialize our true probabilistic beliefs.”
Probability exist as a concept in our heads just like god and math does.
Probability can be used to say something about closed systems like a game of poker using a deck of cards.
If 4 aces have been used I know that there will be no more aces dealt. I can explain why there won't be another ace. And even if magically there was one, we can explain that someone cheated or an extra card was accidentally in the deck.
If 3 aces have been dealt and there are 10 cards left there is statistically a 10% chance of me getting an ace, however I have no way of knowing. The card has already been dealt and I will either get it or I wont.
If I play 10 games will I be getting it 1 time? if I play 100 will i be getting i 10 times? I have no way of knowing.
Now if I learn how to read you and know that you blink more often if you get an ace then I have found an explanation for why I might fold. Not based on statistics but because I can read you. I might still be wrong but I have an explanatory model not based on statistics to play against you, one which I might have to adjust if it turned out to be wrong (ex. you know I know so you change behavior) but at least it's providing me with something more tangible (you give clues I might be able to learn how to interpret)
With the probability it's purely based on blind belief, not substantiated by reality.
Now a deck of card is isolated and not really consequential to society.
But now apply this to open systems economics, weather, climate, elections etc. and you will start to see the problems with assuming there is casual connection when you end up turning that into policy.
The thing is that I'm not sure of what you are saying.
Can physics explain that if you put together a glass of water and a glass of ethanol you end up with a mixture of water and ethanol?
Do you agree that it's possible - but unlikely - that they don't mix?
For other comments of yours it seems that your conclusion would be "I have no way of knowing. It depends on the positions and velocities of each of the molecules and they will either mix or they won't."
I really would appreciate having a clear answer to the question "Can physics explain that if you put together a glass of water and a glass of ethanol you end up with a mixture of water and ethanol?". Don't be shy to answer "no" if that's your idea of physics!
I don't understand the answer. (Assuming it's "yes".)
What is the explanation of why they will mix? Does it involve probabilities?
Why didn't you say "I have no way of knowing. It depends on the positions and velocities of each of the molecules and they will either mix or they won't."
(By the way, if I had asked about a container of neon and a container of helium would you have also claimed that asking whether they will mix is a chemistry question?)
So, would you claim that: "There is no probability whether an *electron* is on the path towards earth. It either is or it isn't."? I guess you must know something that Heisenberg didn't.
Good luck defining things such as "path" and "position", without using probabilities, for non-macroscopic objects.
Also, what is your opinion about the classical "double-slit experiment"? Either a particle passes through a slit, or it passes though the other, right?
What you're saying is irrefutable in the sense that you can define things as you please!
I was pointing out that at the Royal Swedish Academy of Sciences they are not aware that physics doesn't deal with probability and have given the Nobel Prize in Physics this year to some people who write the kind of books and papers that apparently you consider wrong - full of probabilities of transition, detection and whatnot.
You don't get a nobel price for truth. You get a nobel price for significant contributions to the field you are in. Those can be wrong, plenty of noble price winners have been wrong.
> In reality it either is or it isn't. If you have good explanations like we do in physics you don't need probability.
You mean good explanations and observations? You're absolutely right in that if you are able to observe all the relevant information with no noise, you don't need probability. But there are a lot of systems where you can't noiselessly observe what you want – this is where probability is important.
So if I asked you whether you found it more likely that you'll ride a helicopter tomorrow, or that a Democrat will be elected president in the next US presidential election, what would be your answer and why?
I would say "I don't know" and I would be lying if I said anything else.
I can tell you I don't have any plans of going on a helicopter ride tomorrow.
I can also tell you that I have no idea if the next president will be democrat.
What formula do you propose we use to calculate the probability?
And if I proposed a bet where you pay me $10 now and I pay you $40 if there's a Democrat president next election, you wouldn't take the bet because "you don't know?"
(This situation could be put in a less abstract way: there's a business opportunity that costs some money to realise but you only reap the benefit in the right political climate.)
I might take the bet but I would just be guessing or I might work towards trying to find a solution to turn the guess into an explanation.
At no time does probability help me with figuring out what the outcome is.
I can either explain what will happen or I can't. There is no 50/50. I just don't have the correct explanation i.e. an explanation that is hard to vary.
Can I interpret your taking the bet as an admission that it's a bet that comes with a positive expectation? (In the sense that if you took similar bets very many times, you would end up with an almost sure profit.)
Yes and often I don't end up using it. It will either rain where I am or it wont. I might as well believe in Thor the Thunder God as a theory for why I will bring an umbrella.
Every single day it will either rain or it won't. Sometimes you will take an umbrella. Based on what you consider the X of rain. That action of yours will be - you hope - non-causally correlated with reality in the sense that if you take the umbrella very many times, you would end up using it often enough to make it worth it.
X is what people call 'probability'. How do you call it? [Thor's will?] Have you noticed any correlation between you taking the umbrella and rain later in the day?
I don't know what does it mean to "require probability".
I'd say that it "is probability". The concept that is in our heads, mind you.
When you are guessing between the exhaustive and mutually exclusive outcomes A, B and C in such a way that
guess(A)>0, guess(B)>0, guess(C)>0
and
guess(A or B or C) = guess(A) + guess(B) + guess(C) = 1
and
guess(A or B) = guess(A) + guess(B), guess(A or C) = guess(A) + guess(C), guess(B or C) = guess(B) + guess(C)
then
guess is a probability measure as defined by Kolmogorov, for example.
That's why most people don't see any problem in calling that "probability".
You may like the name of "guessing" better but insisting in that it should not be referred to as probability and it doesn't require probability seems an uphill battle.
If you see no difference between saying "it either rains or it doesn't" and saying that some days it's more likely to rain than other days I wonder why would you ever decide take an umbrella with you in case it rains later.
But that's fine! This line of discussion seems exhausted.
You don't make a probabilistic calculation based on some formula to figure out whether you bring your umbrella with you or not.
You just guess, sometime based on past experience.
Sometime you are right, sometimes you are wrong. But you are always just guessing and no formula will allow you to end up with whatever probability you calculate.
Please correct me if I'm wrong. I may have misunderstood you. You say that:
A) there is no difference between saying "It either happens or it doesn't" and "There is a 40% chance it happens"
and, I assume, you also say that:
B) there is no difference between saying "It either happens or it doesn't" and "There is a 60% chance it happens"
So - unless you also have your own ideas of how logic works - you will also say that:
C) there is no difference between saying "There is a 40% chance it happens" and "There is a 60% chance it happens"
If there is no difference how could it be based on past experience or whatever?
---
But maybe you didn't mean that and you recognize that saying that the probability is 60% means saying that it's more probable than when it's said that the probability is 40%.
It's clear that you don't like people using the word probability. However, the fact is that this use is completely in line with the mathematical definition of probability and with the meaning and etymology of "probable" (likely, reasonable, plausible, having more evidence for than against).
Sorry, Feynman, whatever else he did, on probability he gets a grade of flat F. Here is why: In his book Lectures on Physics he states that a particle of unknown location has a probability density uniform over all of space. No it doesn't. No such density can exist. Done. Grade flat F.
I tried to be a physics major but could not swallow all the daily really stupid mistakes such as this one by Feynman I got each physics lecture and I didn't have time both to learn the physics AND to clean up the sloppy math. So, I majored in math.
As I learned the math, from some of the best sources, I came to understand just how just plain awful the math of the physics community is.
Then in one of the Adams lectures on quantum mechanics at MIT I saw some of the reason: The physics community takes pride in doing junk math. They get by with it because they won't take the math seriously anyway, that is, they insist on experimental evidence. So, to them, the math can be just a heuristic, a hint for some guessing.
Students need to be told this, in clear terms, early on.
It went on this way: In one of the lectures from MIT a statement was that the wave functions were differentiable and also continuous. Of COURSE they are continuous -- every differentiable function is continuous.
The lectures made a total mess out of correlation and independence. It looks like Adams does not understand the two or their difference clearly.
There was more really sloppy stuff around Fourier theory. I got my Fourier theory from three of W. Rudin's books. It looks like at MIT they get Fourier theory from a comic book.
I got sick, really sick, of the math in physics. Feynman on probability is just one example.
I used to work in an area of applied probability where some statistical-mechanics principles were applicable. I'd read papers where authors were making analogies of a large neural network to a stat-mech system, using an applicable stat-mech approximation, and then differentiating that approximation to get a probability bound.
It gave interesting results, and did show you something about the original problem that was hard to get by sticking to the original formalism. But at the end of the day, you really would not bet the farm on the truth of those approximations...
On the other hand, Fourier analysis was originally doubted and scorned by mathematics, but (if I'm remembering the story correctly) ended up being used so much that theory was developed to explain in what sense the Fourier transform approximates the original function.
Another example of the interplay between physics and mathematics is the percolation problem, where there was a kind of archipelago of physics-motivated results that probabilists have been trying to tidy up for decades now. E.g., sec. 1.2 of: https://www.unige.ch/~duminil/publi/2018ICM.pdf
There is both Fourier series and the Fourier transform. For each, need to establish that they exist. Then will usually want to know that the inverse transform DOES approximate the original function. Details are in 4 books by W. Rudin. One of these, on groups, I ignored. I did too much group theory, e.g., wrote my honors paper on it, and didn't want to do more.
Principles does Fourier series and does not use measure theory. Real and Complex Analysis does the Fourier transform and uses the Lebesgue integral, that is, measure theory. Functional Analysis does Fourier theory with distributions.
I had serious courses from Principles and R & CA. Also Royden and Neveu. All from a star student of Cinlar, long at Princeton. I read FA, quickly. I don't much care to bother with distributions.
I got into applications via the fast Fourier transform and power spectral estimation as in Blackman and Tukey.
There is an intuitive view that sort of works: The given function is a point in a vector space, with an inner product. The sine waves are coordinate axes. They are orthogonal. The Fourier things are projections of the point onto the coordinate axes. The inverse Fourier thing reconstructs the original function. If use only some of the coordinate axes, then get a least squares approximation of the original function. This all works exactly in ordinary linear algebra, e.g., as in Halmos, Finite Dimensional Vector Spaces, sometimes given to physics students studying quantum mechanics.
But these Fourier things have infinitely many coordinate axes, countably infinite for Fourier series and uncountably infinite for the transform, and there the finite dimensional things don't always work. So, Rudin has to be very careful in presenting what does work and proving it -- so with the details it's not easy reading. What fails is a fairly general situation in a fully general Hilbert space. E.g., are not locally compact, but can get some help from a clever use of the parallolgram inequality (somewhat relevant in my startup).
I'm no teacher. I do not now nor have I ever had any desire to be a prof.
For more, get copies of Rudin's books and dig in.
Wonder of wonders, not all physics profs have done that.
There is both Fourier series and the Fourier transform. For each, need to establish that they exist. Then will usually want to know that the inverse transform DOES approximate the original function. Details are in 4 books by W. Rudin. One of these, on groups, I ignored. I did too much group theory, e.g., wrote my honors paper on it, and didn't want to do more.
Principles does Fourier series and does not use measure theory. Real and Complex Analysis does the Fourier transform and uses the Lebesgue integral, that is, measure theory. Functional Analysis does Fourier theory with distributions.
I had serious courses from Principles and R & CA. Also Royden and Neveu. All from a star student of Cinlar, long at Princeton. I read FA, quickly. I don't much care to bother with distributions.
I got into applications via the fast Fourier transform and power spectral estimation as in Blackman and Tukey.
There is an intuitive view that sort of works: The given function is a point in a vector space, with an inner product. The sine waves are coordinate axes. They are orthogonal. The Fourier things are projections of the point onto the coordinate axes. The inverse Fourier thing reconstructs the original function. If use only some of the coordinate axes, then get a least squares approximation of the original function. This all works exactly in ordinary linear algebra, e.g., as in Halmos, Finite Dimensional Vector Spaces, sometimes given to physics students studying quantum mechanics.
But these Fourier things have infinitely many coordinate axes, countably infinite for Fourier series and uncountably infinite for the transform, and there the finite dimensional things don't always work. So, Rudin has to be very careful in presenting what does work and proving it -- so with the details it's not easy reading. What fails is a fairly general situation in a fully general Hilbert space. E.g., are not locally compact, but can get some help from a clever use of the parallolgram inequality (somewhat relevant in my startup).
I'm no teacher. I do not now nor have I ever had any desire to be a prof.
For more, get copies of Rudin's books and dig in.
Wonder of wonders, not all physics profs have done that.
> Sorry, Feynman, whatever else he did, on probability he gets a grade of flat F. Here is why: In his book Lectures on Physics he states that a particle of unknown location has a probability density uniform over all of space. No it doesn't. No such density can exist. Done. Grade flat F.
I would rather consider that because it seems that you "need" a uniform distribution for a particle of unknown location, it might makes sense for such applications from physics to weaken the property that a probability measure has to be σ-additive to that a probability measure has to be additive. Then it should be possible to define such a "uniform probability 'measure' over all space", perhaps similarly to the example given at
Of course, the space was not made explicit. With R the set of real numbers, the usual assumption in the physics is that the math is done in R^3 with the usual inner product, norm, metric, and topology.
The section starts saying that "We want now to talk a little bit about the behavior of probability amplitudes in time. [...] We are always in the difficulty that we can either treat something in a logically rigorous but quite abstract way, or we can do something which is not at all rigorous but which gives us some idea of a real situation—postponing until later a more careful treatment. With regard to energy dependence, we are going to take the second course. We will make a number of statements. We will not try to be rigorous—but will just be telling you things that have been found out, to give you some feeling for the behavior of amplitudes as a function of time. "
> ... the usual assumption in the physics is that the math ...
Soooo, you got into the physics. I avoided getting into the physics. E.g., when Newton wrote out his second law or Maxwell wrote out his equations, it was just math and the assumption was as I mentioned R^3. Stokes theorem, the Navier-Stokes equations were implicitly in R^3.
When physicists talk math concepts, e.g., probability densities, differentiability, continuity, probability, inner products, Fourier transforms, solutions to differential equations, unitary transformations, etc. they are talking math. They should get the math correct.
So, roughly the paradigm might be -- as a student I assumed it was -- start with a physics problem, convert to a math problem, see what the math says, e.g., solution to a differential equation, Stokes theorem, the weak law of large numbers, then convert back to physics to see what those math results say about the physics.
The idea of a finite universe, say, with a boundary, is popular in popular science and maybe science fiction but is missing from nearly all serious, accepted physics as taught today. So, really, the idea of a finite universe is no good as a patch up of what Feynman wrote.
A simple approach for some of this issue is just to say,
"I have a lab. A neutron is wandering around in it. I don't know where it is. So, I assume the probability density of its location is uniform in the lab."
That's fine, that is, in the math.
Notice, I never used rigorous. When physics profs used that word, it meant that it was time for me, no delay, don't wait for the end of the class period, just to stand, say nothing, walk out, drop the course, and return to the math department.
"If the uncertainty in momentum is zero, the uncertainty relation, ΔpΔx=ℏ, tells us that the uncertainty in the position must be infinite" doesn't make sense except when zero and infinity are understood as limits, for example.
Whenever I see that in a physics video on YouTube, one more such outrage and it's back to a Marilyn Monroe movie. Same for a book on quantum mechanics.
If they have an application of Parseval's theorem, then be clear about it and trot it out. But usually the
Δp and Δx
are not defined clearly or defined at all. I just can't take such sloppiness seriously.
> really stupid mistakes such as this one by Feynman
It's not a mistake, it's a "lie-to-children" fundamentally no different from an intro analysis class talking about "the" real numbers. Freshmen aren't ready for model theory, and they're not ready for rigged Hilbert spaces.
The set of real numbers is commonly defined and their properties established in a first course in abstract algebra. Model theory is not required. Axiomatic set theory is not even required.
Hilbert space? A complete inner product space where complete means every Cauchy convergent sequence converges. The real numbers serve as one example. With the usual inner product, R^3 (where R is the set of real numbers) is another example.
> The set of real numbers is commonly defined and their properties established in a first course in abstract algebra.
A set modelling the real numbers is defined in introductory math courses. The process of quotienting out all the set-theoretic slag to arrive at the real numbers, singular, is generally skipped. As well it should be: it's too ugly to be an end in itself, and too subtle to be of use to freshmen.
> Hilbert space? A complete inner product space where complete means every Cauchy convergent sequence converges.
I said rigged Hilbert spaces. The proper setting for QM is a Hilbert space equipped with dense subspace P of test functions (for which the inclusion P -> H is continuous). This induces an inclusion of dual spaces H* -> P*, and Reisz-Frechet gives us a natural isomorphism H ~ H*, so we have the "Gelfand triple": P -> H -> P*.
The view from 10,000 feet here is that P describes the space of possible measurements, P* describes the distributions, and the fact that the inner product <P, P*> factors through H is a compactness condition. So in particular, a plane wave |k> lives in P*, not H.
But you can't teach this to introductory physics students. They won't have the necessary math background for several years. (Many of them will never have it). And yet they have to be taught.
Thanks for the clarification. I never heard of rigged in either math or physics.
I've tried to understand quantum mechanics, but apparently I keep getting YouTube MIT lectures and books that make a mess out of the subject. E.g., they keep saying that the wave functions, which are differentiable, form a Hilbert space. No they don't: They fail completeness. I have no patience with that crap-ola.
First, you glazed right over the 'rigged' in hither_shore's original post. Did you just not see it or did you think it was some kind of meaningless term to be ignored? Either way, in the context of 'really stupid mistakes' it stands out.
Second, it's not very productive to be so uncharitable when reading technical work. As a reader, you can either assume 'the author is an idiot, I'm going to stop here' or 'the author has elided some details, and they will probably not be relevant later, so I will intuit what they should have said and carry on'.
In this particular case, I think it's clear that you should have interpreted 'Hilbert space of differentiable functions' as something like 'the completion of the pre-Hilbert space of differentiable functions' aka 'the smallest Hilbert space containing the differentiable functions'.
For your "pre-Hilbert" space, I never saw that either but likely my guess at what you mean would be correct.
Can go ahead and take the completion (maybe the same as what used to be called the "normal completion"), but the result will have a lot of functions that are not differentiable and, thus, not quantum mechanics wave functions. So, I would be reluctant to take the resulting actual Hilbert space as what physics was talking about.
If we are going to talk seriously about a subject as serious as quantum mechanics, e.g., to try to understand what is known and maybe to try to clean up what is commonly called its many loose ends and maybe make progress in some applications, e.g., condensed matter physics, super conductivity, maybe even something as common as just why a grid of wire reflects radar waves, maybe connecting with gravity, dark energy, the first second, the Casimir effect, Hawking radiation, and I would include entanglement, then IMHO we need to be as clear as possible on the definitions, theorems, proofs, passing between physics and pure math, etc. After all, often in the research, we are trying to replace some of what is accepted or find what in the past was not discovered. When I see sloppy, I'm not making progress but looking at some at least some little research problems.
> but the result will have a lot of functions that are not differentiable and, thus, not quantum mechanics wave functions. So, I would be reluctant to take the resulting actual Hilbert space as what physics was talking about.
Wavefunctions aren't differentiable functions, because they're not functions. They're distributions - described in general by the rigged Hilbert space structure I described previously, and in the simplified normalizable case typical of introductory courses by L^2(R^n).
> to try to understand what is known and maybe to try to clean up what is commonly called its many loose ends and maybe make progress in some applications, e.g., condensed matter physics, super conductivity, maybe even something as common as just why a grid of wire reflects radar waves, maybe connecting with gravity, dark energy, the first second, the Casimir effect, Hawking radiation, and I would include entanglement, then IMHO we need to be as clear as possible on the definitions, theorems, proofs, passing between physics and pure math, etc.
Ok, but that's not what undergraduate textbooks are for. If you want mathematical physics without any pedagogical concessions, go read mathematical physics papers.
Yes, the set of reals, as in the usual construction, is the only complete Archimedean ordered field.
And some amazing properties are (with my TeX markup) in
John C.\ Oxtoby,
{\it Measure and Category:\ \
A Survey of the Analogies between
Topological and Measure Spaces,\/}
ISBN 3-540-05349-2,
Springer-Verlag,
Berlin,
1971.\ \
One counterargument is that space is finite, and so your choice of n is greater than the volume of the universe. (And so sigma-additivity doesn't apply, since your choices of cubic inches are not disjoint.)
But sure, if you're assuming an unbounded space with finite measure, a uniform density across that space must be identically zero everywhere.
graycat's pedantic approach to mathematical formalism is quite defeatist in that it basically disallows any mathematical concept to advance from precise but limited language towards the edge of imagination. The kind that forbids sqrt(2) from existing in the Pythagorean days.
A probability theory that accommodates the concept of "picking a random even number from all integers" can be valuable, isn't compatible with measure theory but is easy to grasp intuitively. When the mathematical tools aren't good enough you still want to be able to reason about concepts, which is why the tools are developed to begin with. Fortunately almost all things are conceptually tractable when dealing with finite space or quantities; if a statement can be transformed to something rigorous (if numerically imprecise) by forcing boundedness it's not too terrible to speak in unbounded terms when the physical world is what you want to model. I can understand why physicists don't want to be burdened with too much about rigor - they can afford the small risk they're wrong sometimes, but can't afford to slow down their search for new discoveries, when so many questions remain unanswered.
Here we see another problem, the use of random. At the level of Kolmogorov, Loeve, Breiman, Neveu, Cinlar, Karr, etc., we use random only in random variable. We define random variable carefully but never say what random by itself means.
The MIT quantum mechanics lecture I mentioned keeps using random -- struggles with it, says it over and over as if that would make it more clear. In that lecture, the meaning of random is not clear.
What that prof means is likely
"the values of some random variables that are independent and identically distributed (i.i.d.)."
Then we can have some random variables that take values only on the positive, even integers. But, sure, they can't be uniformly distributed. So, with the values of such random variables, we can get some even integers selected independently.
In good treatments of probability, e.g., (from my bibliography file, with TeX markup)
Jacques Neveu,
{\it Mathematical Foundations of the Calculus of Probability,\/}
Holden-Day,
San Francisco.
independence is very nicely developed, e.g., to handle even a set of uncountably infinitely many independent random variables.
As I recall, Neveu was a Loeve student.
That book my Neveu is my candidate for the most elegantly written math book of all time.
But the development of independence is not trivial.
I would expect that independence is needed at times in physics, especially in quantum mechanics, but that only a small fraction of physics profs have studied independence much like what is in Neveu. One result is some confusion between independent and uncorrelated. When I'm watching a physics lecture and see such confusion, I stop taking the physics prof seriously and turn to something more serious, say, an old Marilyn Monroe movie!
This derivation is in the context of classical field theory, but QED is only a short hop away through path integrals.
It’s quite remarkable how the complexity of Maxwell’s equations can be reduced to a single term in the Lagrangian - (F_uv)(F^uv), assuming no charges. That’s really it!
Classical behavior emerges from quantum mechanics as you enter the classical domain.
This can be explained through phase decoherence. As temperature rises, random phase shifts are introduced, which effectively removes the quantum effect. You can show mathematically how this works.
Where P(ϕ) is a probability function for the phase shift. If the probability function is flat, the integral is zero since you're integrating the cosine across its domain. What you get is the classical result!
2 2
|<X|ψ>| = |ψ(X,t)|
You can even re-phrase random phase shifts into a diffusion equation, and find that given α as the diffusion coefficient
2 2 1 -αt 2 pXa
|<X|ψ>| = |ψ(X,t)| - (1 + e cos (--- + ϕ) )
2 ħL
i.e. the transition behavior from quantum to classical dependent on a direct measure of the decoherence!
α small => quantum result, α = large, classical result.
This was much more common in the first 2/3 of the 20th century than it is today. I can strongly recommend Theory of Probability (de Finetti, compiled 1970 based on work de Finetti did as early as 1930s) and Foundations of Statistics (Savage, 1972) – the latter leans a bit on the former but expands on it with useful perspectives.
I recommend you start with these basic theoretical books to get a sense of what it's all built on. But then if you want more practical advice about how to handle things, books on sampling theory tend to hit a sweetspot between theory and practise, in my experience. I like Sampling Techniques (Cochran, 1953) and Sampling of Populations (Levy & Lemeshow, 2013).
https://quantumfrontiers.com/2018/12/23/chasing-ed-jayness-g...
Jaynes about Probability in Science:
https://www.cambridge.org/gb/academic/subjects/physics/theor...