In statistical mechanics, a semi-classical derivation of the entropy that does not take into account the indistinguishability of particles, yields an expression for the entropy which is not extensive (is not proportional to the amount of substance in question). This leads to a paradox known as the Gibbs paradox, after Josiah Willard Gibbs. The paradox allows for the entropy of closed systems to decrease, violating the second law of thermodynamics. A related paradox is the "mixing paradox". If one takes the perspective that the definition of entropy must be changed so as to ignore particle permutation, the paradox is averted.
- 1 Illustration of the problem
- 2 Calculating the entropy of ideal gas, and making it extensive
- 3 The mixing paradox
- 4 Entropy of two ideal gases with particle exchange
- 5 References
- 6 Further reading
- 7 External links
Illustration of the problem
Gibbs himself considered the following problem that arises if the ideal gas entropy is not extensive. Two identical containers of an ideal gas sit side-by-side. There is a certain amount of entropy S associated with each container of gas, and this depends on the volume of each container. Now a door in the container walls is opened to allow the gas particles to mix between the containers. No macroscopic changes occur, as the system is in equilibrium. The entropy of the gas in the two-container system could be immediately calculated, but if the equation is not extensive, the entropy would not be 2S. In fact, Gibbs' non-extensive entropy equation would predict additional entropy. Closing the door then reduces the entropy again to 2S, in supposed violation of the Second Law of Thermodynamics.
As understood by Gibbs, and reemphasized more recently, this is a misuse of the entropy equation. If the gas particles are distinguishable, closing the doors will not return the system to its original state - many of the particles will have switched containers. There is a freedom in what is defined as ordered, and it would be a mistake to conclude the entropy had not increased. In particular, Gibbs' non-extensive entropy equation of an ideal gas was not intended for varying numbers of particles.
The paradox is averted by concluding the indistinguishability (at least effective indistinguishability) of the particles in the volume. This results in the extensive Sackur–Tetrode equation for entropy, as derived next.
Calculating the entropy of ideal gas, and making it extensive
In classical mechanics, the state of an ideal gas of energy U, volume V and with N particles, each particle having mass m, is represented by specifying the momentum vector p and the position vector x for each particle. This can be thought of as specifying a point in a 6N-dimensional phase space, where each of the axes corresponds to one of the momentum or position coordinates of one of the particles. The set of points in phase space that the gas could occupy is specified by the constraint that the gas will have a particular energy:
and be contained inside of the volume V (let's say V is a box of side X so that V=X3):
for : and
The first constraint defines the surface of a 3N-dimensional hypersphere of radius (2mU)1/2 and the second is a 3N-dimensional hypercube of volume VN. These combine to form a 6N-dimensional hypercylinder. Just as the area of the wall of a cylinder is the circumference of the base times the height, so the area φ of the wall of this hypercylinder is:
The entropy is proportional to the logarithm of the number of states that the gas could have while satisfying these constraints. In classical physics, the number of states is infinitely large, but according to quantum mechanics it is finite. Before the advent of quantum mechanics, this infinity was regularized by making phase space discrete. Phase space was divided up in blocks of volume . The constant h thus appeared as a result of a mathematical trick and thought to have no physical significance. However, using quantum mechanics one recovers the same formalism in the semi-classical limit, but now with h being Planck's constant. One can qualitatively see this from Heisenberg's uncertainty principle; a volume in N phase space smaller than h3N (h is Planck's constant) cannot be specified.
To compute the number of states we must compute the volume in phase space in which the system can be found and divide that by . This leads us to another problem: The volume seems to approach zero, as the region in phase space in which the system can be is an area of zero thickness. This problem is an artifact of having specified the energy U with infinite accuracy. In a generic system without symmetries, a full quantum treatment would yield a discrete non-degenerate set of energy eigenstates. An exact specification of the energy would then fix the precise state the system is in, so the number of states available to the system would be one, the entropy would thus be zero.
When we specify the internal energy to be U, what we really mean is that the total energy of the gas lies somewhere in an interval of length around U. Here is taken to be very small, it turns out that the entropy doesn't depend strongly on the choice of for large N. This means that the above "area" must be extended to a shell of a thickness equal to an uncertainty in momentum , so the entropy is given by:
This quantity is not extensive as can be seen by considering two identical volumes with the same particle number and the same energy. Suppose the two volumes are separated by a barrier in the beginning. Removing or reinserting the wall is reversible, but the entropy difference after removing the barrier is
which is in contradiction to thermodynamics. This is the Gibbs paradox.
The paradox is resolved by postulating that the gas particles are in fact indistinguishable. This means that all states that differ only by a permutation of particles should be considered as the same state. For example, if we have a 2-particle gas and we specify AB as a state of the gas where the first particle (A) has momentum p1 and the second particle (B) has momentum p2, then this state as well as the BA state where the B particle has momentum p1 and the A particle has momentum p2 should be counted as the same state.
For an N-particle gas, there are N! states which are identical in this sense, if one assumes that each particle is in a different single particle state. One can safely make this assumption provided the gas isn't at an extremely high density. Under normal conditions, one can thus calculate the volume of phase space occupied by the gas, by dividing Equation 1 by N!. Using the Stirling approximation again for large N, ln(N!) ≈ N ln(N) - N, the entropy for large N is:
which can be easily shown to be extensive. This is the Sackur–Tetrode equation.
The mixing paradox
A closely related paradox is the mixing paradox. Again take a box with a partition in it, with gas A on one side, gas B on the other side, and both gases are at the same temperature and pressure. If gas A and B are different gases, there is an entropy that arises due to the mixing. If the gases are the same, no additional entropy is calculated. The additional entropy from mixing does not depend on the character of the gases. The paradox is that the two gases can be arbitrarily similar, but the entropy from mixing does not disappear unless they are the same gas.
The resolution is provided by a careful understanding of entropy. In particular, as explained concisely by Jaynes, there is an arbitrariness in the definition of entropy.
A central example in Jaynes' paper relies on the fact that, if one develops a theory based on the idea that the two different types of gas are indistinguishable, and one never carries out any measurement which detects this fact, then the theory will have no internal inconsistencies. In other words, if there are two gases A and B and we have not yet discovered that they are different, then assuming they are the same will cause no theoretical problems. If ever an experiment is performed with these gases that yields incorrect results, we will certainly have discovered a method of detecting their difference and recalculating the entropy increase when the partition is removed.
This insight suggests that the idea of thermodynamic state and entropy are somewhat subjective. The differential increase in entropy (dS), as a result of mixing dissimilar element sets (the gases), multiplied by the temperature (T) is equal to the minimum amount of work we must do to restore the gases to their original separated state. Suppose that the two different gases are separated by a partition, but that we cannot detect the difference between them. We remove the partition. How much work does it take to restore the original thermodynamic state? None – simply reinsert the partition. The fact that the different gases have mixed does not yield a detectable change in the state of the gas, if by state we mean a unique set of values for all parameters that we have available to us to distinguish states. As soon as we become able to distinguish the difference, the amount of work necessary to recover the original macroscopic configuration becomes non-zero, and the amount of work does not depend on the magnitude of that difference.
This line of reasoning is particularly informative when considering the concepts of indistinguishable particles and correct Boltzmann counting. Boltzmann's original expression for the number of states available to a gas assumed that a state could be expressed in terms of a number of energy "sublevels" each of which contain a particular number of particles. While the particles in a given sublevel were considered indistinguishable from each other, particles in different sublevels were considered distinguishable from particles in any other sublevel. This amounts to saying that the exchange of two particles in two different sublevels will result in a detectably different "exchange macrostate" of the gas. For example, if we consider a simple gas with N particles, at sufficiently low density that it is practically certain that each sublevel contains either one particle or none (i.e. a Maxwell–Boltzmann gas), this means that a simple container of gas will be in one of N! detectably different "exchange macrostates", one for each possible particle exchange. Just as the mixing paradox begins with two detectably different containers, and the extra entropy that results upon mixing is proportional to the average amount of work needed to restore that initial state after mixing, so the extra entropy in Boltzmann's original derivation is proportional to the average amount of work required to restore the simple gas from some "exchange macrostate" to its original "exchange macrostate". If we assume that there is in fact no experimentally detectable difference in these "exchange macrostates" available, then using the entropy which results from assuming the particles are indistinguishable will yield a consistent theory. This is "correct Boltzmann counting". It is often said that the resolution to the Gibbs paradox derives from the fact that, according to the quantum theory, like particles are indistinguishable in principle. By Jaynes' reasoning, if the particles are experimentally indistinguishable for whatever reason, Gibbs paradox is resolved, and quantum mechanics only provides an assurance that in the quantum realm, this indistinguishability will be true as a matter of principle, rather than being due to an insufficiently refined experimental capability.
Entropy of two ideal gases with particle exchange
A purely classical calculation of the entropy of an ideal gas yields an extensive result for two systems if they are allowed to exchange particles. The following calculation simplifies this calculation in three ways:
- The ideal gas consists of particles that confined to one spatial dimension.
- Terms are neglected that become important when n is not very large. These terms may be neglected in the limit that , but nevertheless are important, for example in deriving the Sackur–Tetrode equation. These neglected terms are also likely to be important in computer simulation and nanotechnology where the number of particles is not very large.
- The subdivision of phase space into units of Planck's constant (h) is omitted. Instead entropy is taken as an integral over available phase space. This serves to highlight the purely classical nature of the calculation.
The integral is restricted to a contour of available regions of phase space, subject to conservation of energy. In contrast to the one-dimensional line integrals students encounter in college physics, the contour of constant energy possesses a vast number of dimensions. The justification for integrating over canonical phase space involves the assumption of equal probability. The assumption can be made by invoking the ergodic hypothesis (not always valid in computer simulations and actual experiments.) as well as the Liouville's theorem (mathematically provable for Hamiltonian systems.) Liouville's theorem assumes a fixed number of dimensions that the system 'explores'. In most calculations of entropy, the number dimensions equals the number of particles in the system, which forces phase space to abruptly change dimensionality when particles are added or subtracted. This may explain the difficulties in constructing a clear and simple derivation for the dependence of entropy on the number of particles.
For the ideal gas, the accessible phase space is an n-sphere (also called a hypersphere) in the dimensional space:
To recover the paradoxical result that entropy is not extensive, we integrate over phase space for a gas of monatomic particles confined to a single spatial dimension by . Since expressions for classical entropy are well known, and our goal is to resolve a paradox, we simplify notation by taking both the particle's mass and Boltzmann's constant equal to unity: . Since the gas is one-dimensional, we may use the vector symbol to denote N and 2N dimensional vectors:
- where and
To calculate entropy, we use the fact that the n-sphere, has a hypersurface of,
Note that this "surface" is actually a dimensional volume. If n = 2, the n-sphere is a circle, and the hypersurface, , is the (one-dimensional) circumference. If n is an odd number, it is necessary use the gamma function to evaluate the factorial.
Gibb's paradox in a one-dimensional gas
Gibb's paradox arises when entropy is calculated using an dimensional phase space, where is also the number of particles in the gas. These particles are spatially confined to the one-dimensional interval :
The subscripts on are used to define the 'state variables' and will be will be discussed later, when it is argued that the number of particles, lacks full status as a state variable in this calculation. The integral over configuration space is . As indicated by the underbrace, the integral over velocity space is restricted to the "surface area" of the n dimensional hypo-sphere of radius , and is therefore equal to the "area" of that hypo-surface:
Click to view the algebraic steps We begin with:
Both terms on the right hand side have dominant terms. Using the Stirling approximation for large M, , we have:
Terms are neglected if they exhibit less variation with a parameter, and we compare terms that vary with the same parameter. Entropy is defined with an additive arbitrary constant because the area in phase space depends on what units are used. For that reason it does not matter if entropy is large or small for a given value of E. We instead to seek how entropy varies with E, i.e., we seek :
- An expression such as is much less important than an expression like
- An expression like is much less important than an expression like . Note that the logarithm is not a strongly increasing function. The neglect of terms proportional to n compared with terms proportional to n ln n is only justified if n is extremely large.
Combining the important terms:
After dropping the small terms:
In the second expression, the term was subtracted and added, using the fact that . This was done to highlight the fact that entropy must be an extensive property of matter. Third term is neither extensive nor intensive and is therefore wrong. The arbitrary constant has been added because entropy can usually be viewed as being defined with an arbitrary additive constant. This is especially true in entropy is defined as the logarithm of a phase space measured in units of momentum-position. Any change in how these units are defined will add or subtract a constant from the value of the entropy.
Alternative ways to make classical entropy extensive
As discussed above, an extensive form of entropy is recovered if we divide the volume of phase space, , by n!. An alternative approach is to argue that the dependence on particle number cannot be trusted on the grounds that changing also changes the dimensionality of phase space. Such changes in dimensionality lie outside the scope of Hamiltonian mechanics and Liouville's theorem. For that reason it is plausible to allow the arbitrary constant to be a function of . Defining the function to be, , we have:
which has extensive scaling:
Integrating phase space
Following Swendsen, we allow two systems to exchange particles. This essentially 'makes room' in phase space for particles to enter or leave without requiring a change in the number of dimensions of phase space. The total number of particles is :
- particles have coordinates .
- The total energy of these particles is
- particles have coordinates .
- The total energy of these particles is
- The system is subject to the constraints, and
Taking the integral over phase space, we have:
The question marks (?) serve as a reminder that we may not assume that that the first nA particles (i.e. 1 though nA) are in system-A while the other particles (nB through N) are in system-B. (This is further discussed in the next section.)
Taking the logarithm and keeping only the largest terms, we have:
This can be interpreted as the sum of the entropy of system-A and system-B, both extensive. And there is a term, , that is not extensive.
Letting N = 3 for three-D visualization
The correct (extensive) formulas for systems A and B were obtained because we included all the possible ways that the two systems could exchange particles. The use of combinations (i.e. N particles choose NA) was used to ascertain the number of ways N particles can be divided into system-A containing nA particles and system_B containing nB particles. This counting is not justified on physical grounds, but on the need to integrate over phase space. As will be illustrated below, phase space contains not a single nA-sphere and a single nB-sphere, but instead
pairs of n-spheres, all situated in the same N+1-dimensional velocity space. The integral over accessible phase space must include all of these n-spheres, as can be seen in the figure, which shows the actual velocity phase space associated a gas that consists of three particles. Moreover, this gas has been divided into two systems, A and B.
If we ignore the spatial variables, the phase space of a gas with three particles is three dimensional, which permits one to sketch the n-spheres over which the integral over phase space must be taken. If all three particles are together, the split between the two gasses is 3|0. Accessible phase space is delimited by an ordinary sphere (2-sphere) with a radius that is either or (depending which system has the particles).
If the split is 2|1, then phase space consists of circles and points. Each circle occupies two dimensions, and for each circle, two points lie on the third axis, equidistant from the center of the circle. In other words, if system-A has 2 particles, accessible phase space consists of 3 pairs of n-spheres, each pair being a 1-sphere and a 0-sphere:
- Gibbs, J. Willard (1875–1878). On the Equilibrium of Heterogeneous Substances. Connecticut Acad. Sci. ISBN 0-8493-9685-9. Reprinted in Gibbs, J. Willard (October 1993). The Scientific Papers of J. Willard Gibbs (Vol. 1). Ox Bow Press. ISBN 0-918024-77-3. and in Gibbs, J. Willard (February 1994). The Scientific Papers of J. Willard Gibbs (Vol. 2). Ox Bow Press. ISBN 1-881987-06-X.
- Jaynes, E.T. (1996). "The Gibbs Paradox" (PDF). Retrieved November 8, 2005.
- Grad, Harold (1961). "The many faces of entropy". Communications on Pure and Applied Mathematics 14 (3): 323. doi:10.1002/cpa.3160140312.
- van Kampen, N. G. (1984). "The Gibbs Paradox". In W. E. Parry. Essays in Theoretical Physics in Honor of Dirk ter Haar. Oxford: Pergamon. ISBN 978-0080265230.
- Swendsen, Robert (March 2006). "Statistical mechanics of colloids and Boltzmann's definition of entropy". American Journal of Physucs 74 (3): 187–190. doi:10.1119/1.2174962.
- Swendsen, Robert H. (June 2002). "Statistical Mechanics of Classical Systems with Distinguishable Particles". Journal of Statistical Physics 107 (5/6): 1143–1166. Bibcode:2002JSP...107.1143S. doi:10.1023/A:1015161825292.
- Jaynes, E.T. (1992). The Gibbs Paradox in Maximum Entropy and Bayesian Methods (edited by C.R. Smith, G.J. Erickson, & P.O. Neudorfere). Dordrecht, Holland: Kluwer Academic Publishers. pp. 1–22.
In particular, Gibbs failed to point out that an 'integration constant' was not an arbitrary constant, but an arbitrary function. But this has, as we shall see, nontrivial physical consequences. What is remarkable is not that Gibbs should have failed to stress a one mathematical point in almost the last words he wrote; but that for 80 years thereafter all textbook writers (except possibly Pauli) failed to see it.
- See both references to Swendsen above
- Chih-Yuan Tseng and Ariel Caticha (2001). R. L. Fry, ed. "Bayesian Inference and Maximum Entropy Methods in Science and Engineering". AIP Conference Proceedings 617 (617). p. 331. arXiv:cond-mat/0109324. doi:10.1063/1.1477057.
- Dieks, Dennis (2011). "The Gibbs Paradox Revisited". In Dennis Dieks, Wenceslao J. Gonzalez, Stephan Hartmann, Thomas Uebel and Marcel Weber. Explanation, Prediction, and Confirmation. The Philosophy of Science in a European Perspective. pp. 367–377. arXiv:1003.0179. doi:10.1007/978-94-007-1180-8_25. ISBN 978-94-007-1179-2.
- Gibbs paradox and its resolutions - varied collected papers