Entropy (thermodynamics)

In thermodynamics, entropy is a measure of a thermodynamic system's disorder. The entropy of the system varies directly with any reversible change in heat and inversely with the net temperature of the system. (The concept of entropy has somewhat different meanings in information theory, economics, and other disciplines.) Entropy is central to the second law of thermodynamics, which states that the entropy of an isolated system left to spontaneous evolution cannot decrease with time. As a result, isolated systems evolve toward thermodynamic equilibrium, where the entropy is highest. A consequence of the second law of thermodynamics is that certain processes are irreversible.

Quotes

• The choice (or accident) of initial conditions creates a sense of time directionality in a physical environment. The 'arrow' of entropy increase is a reflection of the improbability of those initial conditions which are entropy-decreasing in a closed physical system. ...
  Everywhere... in the Universe, we discern that closed physical systems evolve in the same sense from ordered states towards a state of complete disorder called thermal equilibrium. This cannot be a consequence of known laws of change, since... these laws are time symmetric—they permit... time-reverse... The initial conditions play a decisive role in endowing the world with its sense of temporal direction. ...some prescription for initial conditions is crucial if we are to understand... A Theory of Everything needs to be complemented by some such independent prescription which appeals to simplicity, economy, or some other equally metaphysical notion to underpin its credibility. The only radically different alternative... a belief that the type of mathematical description of Nature... —that of causal equations with starting conditions—is just an artefact of our own preferred categories of thought and merely an approximation... At a deeper level, a sharp divide between those aspects of reality that we habitually call 'laws' and... 'initial conditions' may simply not exist.
  • John D. Barrow, Theories of Everything: The Quest for Ultimate Explanation (1991) pp. 38-39.

• The first step came from W. Wien, whose displacement law of 1893 is embodied in the shift of the maximum of spectrum energy density, from red to violet, with increasing temperatures. Wien showed that a universal function of the ratio of temperature to frequency must here be in question. The determination of this universal function was the culmination of the insight and consistent labors of Planck (1900), who by postulating the energy quantum, became the creator of modern thermodynamics; for this energy element is a saucy reality, whose purpose is to stay. It not only tells us all we know of the distribution of energy in the black body spectrum in its thermal relations, but it gives us, indirectly, perhaps the most accurate data at hand of the number of molecules per normal cubic centimeter of the gas, of the mean translational energy of its molecules, of the molecular mass, of the Boltzmann entropy constant, even of the charge of the electron or electric atom itself.
  • Carl Barus, "The Mathematician in Modern Physics" Science (Jul-Dec 1914) Vol. 40, pp. 726-727.

• Of the Planck molecular oscillators... If operating continuously under the established electromagnetic laws they lead to the impossible distributions of energy in the spectrum investigated by Rayleigh and Jeans. But if emitting only, when their energy content is a whole number of energy elements, a case thus involving the entropy probability of Boltzmann, Wien's law and the numerical data referred to are deducible with astounding precision.
  • Carl Barus, "The Mathematician in Modern Physics" Science (Jul-Dec 1914) Vol. 40, p. 727.

• We show that it is natural to introduce the concept of black-hole entropy as the measure of information about a black-hole interior which is inaccessible to an exterior observer. Considerations of simplicity and consistency, and dimensional arguments indicate that the black-hole entropy is equal to the ratio of the black-hole area to the square of the Planck length times a dimensionless constant of order unity. A different approach making use of the specific properties of Kerr black holes and of concepts from information theory leads to the same conclusion, and suggests a definite value for the constant.
  • Jacob Bekenstein, Abstract, "Black Holes and Entropy" Physical Review D (Apr 15, 1973) Vol. 7, Issue 8, p. 2333.

• For well over a hundred years, a basic antithesis was noticed between inanimate and animate nature. The direction of physical events is prescribed by the second principle of thermodynamics... the general trend of physical happenings is toward most probable states, that is, maximum entropy and progressive destruction of differentiation and order. ...The system will tend toward thermal equilibrium ...a state of most probable distribution of molecules ...disappearance of the temperature gradient and uniform distribution ...maximum entropy. "Higher," directed forms of energy (e.g., mechanical, electric, chemical) are dissipated... progressively converted into the lowest form of energy, i.e., undirected heat movement of molecules; chemical systems tend toward equilibria with maximum entropy; machines wear out owing to friction; in communication channels, information can only be lost by conversion of messages into noise but not vice versa, and so forth.
  • Ludwig von Bertalanffy, Robots, Men and Minds' (1967) p. 75.

• If for the entire universe we conceive the same magnitude to be determined, consistently and with due regard to all circumstances, which for a single body I have called entropy, and if at the same time we introduce the other and simpler conception of energy, we may express in the following manner the fundamental laws of the universe which correspond to the two fundamental theorems of the mechanical theory of heat.
  1. The energy of the universe is constant.
  2. The entropy of the universe tends to a maximum.
  • Rudolf Clausius, The Mechanical Theory of Heat: With Its Applications to the Steam-engine and to the Physical Properties of Bodies (1867) ed., Thomas Archer Hirst, Ninth Memoir. On Several Convenient Forms of the Fundamental Equations of the Mechanical Theory of Heat.

• All this prompts the question of why, from the infinite rage of possible values that Nature could have selected for the fundamental constants, and from the infinite variety of initial conditions that could have characterized the primeval universe, the actual values and conditions conspire to produce the particular range of special features that we observe. For clearly the universe is a very special place: exceedingly uniform on a large scale, yet not so precisely uniform that galaxies could not form; extremely low entropy per proton, and hence cool enough for chemistry to happen; almost zero cosmic propulsion and an expansion rate tuned to that energy content to unbelievable accuracy; values for the strengths of its forces that permit nuclei to exist, yet do not burn up all the cosmic hydrogen, and many more apparent accidents of fortune.
  • Paul Davies, The Accidental Universe (1982) p. 111.

• The fundamental problem about trying to define life in terms of physics is easily explained. If you go to a physics department... you'll be given a definition in terms of matter... force... energy... entropy... free energy, molecular binding affinities, and so on. If you go to a biology department... you'll be given a very different narrative in terms of... instructions, transcription, gene editing, translation, coding, signals... Biologists use information-speak... informational qualities... physicists define life in terms of physical quantities.
• Paul Davies, The Demon in the Machine (Sep. 7, 2019) 6th International FQXi Conference, "Mind Matters: Intelligence and Agency in the Physical World." A YouTube video source, 4:31.

• I know of no theorem that tells you... the maximum amount of change that agency can achieve in the universe, and what interests me... is agency at the end of the universe. If you end up in De Sitter space, which has a temperature and a horizon entropy, can you do anything with... those thermal fluctuations? Can you mine them... to extract energy?
• Paul Davies, The Demon in the Machine (Sep. 7, 2019) 6th International FQXi Conference, "Mind Matters: Intelligence and Agency in the Physical World." A YouTube video source, 17:22.

• In the year 1900 Max Planck wrote... ${\displaystyle E=hv}$, where ${\displaystyle E}$ is the energy of a light wave, ${\displaystyle v}$ is its frequency, and ${\displaystyle h}$ is... Plank's constant. It said that energy and frequency are the same thing measured in different units. Plank's constant gives you a rate of exchange for for converting frequency into energy... But in the year 1900 this made no physical sense. Even Plank himself did not understand it. ...Now Hawking has written down an equation which looks rather like Plank's equation... ${\displaystyle S=kA}$, where ${\displaystyle S}$ is the entropy of a black hole, ${\displaystyle A}$ is the area of its surface, and ${\displaystyle k}$ is... Hawking's constant. Entropy means roughly the same thing as the heat capacity of an object. ...Hawking's equation says that entropy is really the same thing as area. The exchange rate... is given by Hawking's constant... But what does it really mean to say that entropy and area are the same thing? We are as far away from understanding that now as Planck was of understanding quantum mechanics in 1900. ...[T]his equation will emerge as a central feature of the still unborn theory which will tie together gravitation and quantum mechanics and thermodynamics.
  • Freeman Dyson, Infinite in All Directions: Gifford Lectures given at Aberdeen, Scotland April-November 1985 (1988) pp. 21-22 (paperback, 1989).

• If I took a heavy weight on the floor here and pushed it, it would slide and stop. ... So, a frictional effect seems to be irreversible. ... a frictional effect ... is the result of enormous complexity of the interaction of the block with the wood ... the jiggling of the atoms inside the wood of the block is changed into disorganized irregular wiggle-waggles of the atoms in the wood.
  • Richard Feynman, (July 11, 2018)"Richard Feynman's Lecture Entropy (Part 01)". EduBloq, YouTube. (quote at 6:47 of 21:31 in video)

• Newton and his theories were a step ahead of the technologies that would define his age. Thermodynamics, the grand theoretical vision of the nineteenth century, operated in the other direction with practice leading theory. The sweeping concepts of energy, heat, work and entropy, which thermodynamics (and its later form, statistical mechanics) would embrace, began first on the shop floor. Originally the domain of engineers, thermodynamics emerged from their engagement with machines. Only later did this study of heat and its transformation rise to the heights of abstract physics and, finally, to a new cosmological vision.
  • Adam Frank, About Time: Cosmology and Culture at the Twilight of the Big Bang (2011)

• Black holes have the universe's most inscrutable poker faces. ...When you've seen one black hole with a given mass, charge, and spin (though you've learned these thing indirectly, through their effect on surrounding gas and stars...) you've definitely seen them all. ...black holes contain the highest possible entropy ...a measure of the number of rearrangements of an object's internal constituents that have no effect on its appearance. ...Black holes have a monopoly on maximal disorder. ...As matter takes the plunge across a black hole's ravenous event horizon, not only does the black hole's entropy increase, but its size increases as well. ...the amount of entropy ...tells us something about space itself: the maximum entropy that can be crammed into a region of space—any region of space, anywhere, anytime—is equal to the entropy contained within a black hole whose size equals the region in question.
  • Brian Greene, The Fabric of the Cosmos (2004)

• A natural guess is that... a black hole's entropy is... proportional to its volume. But in the 1970s Jacob Bekenstein and Stephen Hawking discovered that this isn't right. Their... analyses showed that the entropy... is proportional to the area of its event horizon... less than what we'd naïvely guess. ...Berkenstein and Hawking found that... each square being one Planck length by one Planck length... the black hole's entropy equals the number of such squares that can fit on its surface... each Planck square is a minimal unit of space, and each carries a minimal, single unit of entropy. This suggests that there is nothing, even in principle, that can take place within a Planck square, because any such activity could support disorder and hence the Planck square could contain more than a single unit of entropy... Once again... we are led to the notion of an elemental spatial entity.
  • Brian Greene, The Fabric of the Cosmos (2004)

• [F]or a physicist, the upper limit to entropy... is a critical, almost sacred quantity. ...the Bekenstein and Hawking result tells us that a theory that includes gravity is, in some sense, simpler than a theory that doesn't. ...If the maximum entropy in any given region of space is proportional to the region's surface area and not its volume, then perhaps the true, fundamental degrees of freedom—the attributes that have the potential to give rise to that disorder—actually reside on the region's surface and not within its volume. Maybe... the universe's physical processes take place on a thin, distant surface that surrounds us, and all we see and experience is merely a projection of those processes. Maybe... the universe is rather like a hologram.
  • Brian Greene, The Fabric of the Cosmos (2004)

• Just like a computer, we must remember things in the order in which entropy increases. This makes the second law of thermodynamics almost trivial. Disorder increases with time because we measure time in the direction in which disorder increases. You can’t have a safer bet than that!
  • Stephen Hawking, A Brief History of Time (1988) Ch. 9

• The homeostatic principle does not apply literally to the functioning of all complex living systems, in that in counteracting entropy they move toward growth and expansion.
  • Daniel Katz & Robert L. Kahn (1966), The Social Psychology of Organizations. p. 23.

• Because entropy is not really a classical quantity, we must build quantum mechanics into the definition. ...
  It suffices to define entropy as the logarithm of the number of quantum states accessible to a system.
  • Charles Kittel, (1989). "How to define entropy". Nature 339 (6221). DOI:10.1038/339170a0.

• As the natural sciences have developed to encompass increasingly complex systems, scientific rationality has become ever more statistical, or probabilistic. The deterministic classical mechanics of the enlightenment was revolutionized by the near-equilibrium statistical mechanics of late 19th century atomists, by quantum mechanics in the early 20th century, and by the far-from-equilibrium complexity theorists of the later 20th century. Mathematical neo-Darwinism, information theory, and quantitative social sciences compounded the trend. Forces, objects, and natural types were progressively dissolved into statistical distributions: heterogeneous clouds, entropy deviations, wave functions, gene frequencies, noise-signal ratios and redundancies, dissipative structures, and complex systems at the edge of chaos.
  • Nick Land, "Statistical Mentality" (2011)

• So if we're going to ask... What is life? ...Erwin Schrödinger wrote a famous book on that theme ...Two famous ideas ...emerged ...one ...was ...that genes are a code-script, and that was the first time anybody had used the word "code-script" or really thought in terms of information, in biology. ...This was before DNA was discovered. He was a direct inspiration to Watson and Crick and many others. The second theme... was how life maintains its organization over time, and why don't we just fall to pieces as entropy would tend to suggest... He talked about life feeding on negative entropy, or "negentropy"... [H]e talked about continually sucking order... from its environment. ...[I]t's a wonderful book. ...[H]e said, "If I had been catering for physicists alone I should have let the discussion turn on free energy instead." ...In more modern terms he's saying something like life is the harnessing of chemical energy in such a way that the energy-harnessing device makes a copy of itself. ...[H]e's linking the two key themes of biology ...information and energy together.
  • Nick Lane, Why is life the way it is? (Feb 2, 2017) 15:51 A YouTube video presentation from "The Royal Society" channel.

• The new information technologies can be seen to drive societies toward increasingly dynamic high-energy regions further and further from thermodynamical equilibrium, characterized by decreasing specific entropy and increasingly dense free-energy flows, accessed and processed by more and more complex social, economic, and political structures.
  • Ervin László, "Information Technology and Social Change: An Evolutionary Systems Analysis". Behavioral Science (Oct, 1992) Vol. 37, Issue 4, p. 247.

• There is nothing supernatural about the process of self-organization to states of higher entropy; it is a general property of systems, regardless of their materials and origin. It does not violate the Second Law of thermodynamics since the decrease in entropy within an open system is always offset by the increase of entropy in its surroundings.
  • Ervin László, Introduction to Systems Philosophy: Toward a New Paradigm of Contemporary Thought (1972) p. 44.

• After the invention of the steam-engine... by James Watt, the attention of engineers and of scientific men was directed to... its further improvement. ...Sadi Carnot, in 1824, published Réflexions sur la Puissance Motrice du Feu... [which] examined the relations between heat and the work done by heat used in an ideal engine, and by reducing the problem to its simplest form and avoiding...questions relating to details, he succeeded in establishing the conditions upon which the economical working of all heat-engines depends. ...Though the proof was invalid, the proposition remained true... Carnot's memoir remained for a long time unappreciated, and it was not until use was made of it by William Thomson... in 1848, to establish an absolute scale of temperature, that the merits of the method proposed in it were recognized. ...[H]e found that Carnot's proposition could no longer be proved by denying the possibility of "the perpetual motion," and was led to lay down a second fundamental principle... now called the Second Law of Thermodynamics. ...It was published in March, 1851. In the previous year Clausias published a discussion of the same question... in which he lays down a principle for use in the demonstration of Carnot’s proposition, which, while not the same in form as Thomson’s, is the same in content, and ranks as another statement of the Second Law of Thermodynamics. Clausius followed up this paper by others, and subsequently published a book in which the subject of Thermodynamics was given a systematic treatment, and in which he introduced and developed the important function called by him the entropy.
  • William Francis Magie, Preface to The Second Law of Thermodynamics: Memoirs (1899) p. vi. Magie was editor & translator of these Memoirs of Carnot, Clausius & Thomson

• The most common way to describe entropy is as disorder... associated with things becoming more mixed, random and less ordered, but... the best way to think about entropy is as the tendency of energy to spread out. ...Most of the laws of physics work... the same... forwards or backwards in time. ...So how does this clear time dependence arise? ...[T]his is where Ludwig Boltzmann made an important insight. Heat flowing from cold to hot is not impossible, it's just improbable. ...In everyday solids there are about 100 trillion trillion atoms and even more energy packets, so heat flowing from cold to hot is just so unlikely that it never happens. ...[I]f the ...tendency is to spread out and for things to get messier, then how is it possible to have ...air conditioning, where the cold interior gets cooler and the hot exterior gets hotter? Energy is going from cold to hot, decreasing the entropy of the house. ...[T]his ...is only possible by increasing the entropy a greater amount ...at a power plant ...heating up the environment ...and creating waste heat in the fans and compressor [of the air conditioner]. ...How is there any structure left on earth? ...[I]f the earth were a closed system the energy would spread out completely, meaning all life would cease, everything would decay and mix, and ...reach the same temperature. But luckily the earth is not a closed system, because we have the sun.
  • Derek Muller, "The Most Misunderstood Concept in Physics" (Jul 1, 2023) 11:04, a Youtube video from the Veritasium channel.

• You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage.
  • John von Neumann Scientific American Vol. 225 No. 3, (1971), p. 180. Suggesting to Claude Shannon a name for his new uncertainty function.

• The equations of Newtonian mechanics are reversible in time and Poincaré proved that if a mechanical system is in a given state it will return infinitely often to a state arbitrarily close to the given one. Zermelo deduced that the Second Law of Thermodynamics is impossible in a mechanical system. Boltzmann asserted that entropy increases almost always, rather than always. However he believed that Poincaré's result, although correct in theory, was in practice impossible to observe since the time before a system returns to near its original state was too long.
  • J.J. O'Conner, E.F. Robertson, "Ludwig Boltzmann" (Sept, 1998) Ernst Zermelo ref: "On the Mechanical Explanation of Irreversible Processes"

• Let's talk some energy transfer principles. ...My Grandpappy always used to say "hot goes to cold." ...Things of a higher energy intensity state tend to equalize with things at a lower intensity energy state. ...Where there's differences, things tend towards equilibrium... You put a ball on top of a hill and you give it a chance to roll down the hill, that's what's going to happen... If you leave a big pile of sand outside long enough, it's going to flatten out. ...You take an ice cube and hold it in your hand ...heat goes out of your hand and melts the ice cube until that water becomes the same temperature as your hand. ...[V]oltage tends toward equilibrium ...If you have this ...high voltage [or current] ...stored in a battery ...[T]ake a wire and hook it from one side to the other ...It's going to equalize ...and the battery's going to be dead ...
  • Bryan Orr, Electrical Basics Class (Oct 7, 2022) 27:17. A YouTube video from the HVAC School channel.

• Energy in the universe is constant and it can't be destroyed or created. [1st Law of Thermodynamics] ...You've got what's you've got. You've got entropy, which is a state of disorder, so things tend [from] order to disorder, but you're not going to destroy [the energy].
  ■ Energy goes from organized and usable to disorganized and unusable, and it seeks equilibrium. That's the 2nd law and discusses entropy, which is just decay and disorder and death and destruction and... all that good stuff! All the happy stuff! When you get to be my age, you kind of look forward to it so it's not necessarily that bad.
  ■ Molecular motion stops, as does entropy, at absolute zero. [3rd law] So when you get to absolute zero... nothing moves, which is why we can't really get there... because... we always take heat out of things by putting it into something else [at a lower temperature]...
  ■ Hot goes to cold. (Energy moves from higher temperature to lower temperature.)
  ■ High voltage goes to lower voltage. (Electrical current moves from high potential to low potential.)
  ■ High pressure goes to low pressure.
  • Bryan Orr, Electrical Basics Class (Oct 7, 2022) 28:34. A YouTube video from the HVAC School channel.

• Investigations of the entropy of substances at low temperatures have produced very important information regarding the structure of crystals, the work of Giauque and his collaborators being particularly noteworthy. For example, the observed entropy of crystalline hydrogen shows that even at very low temperatures the molecules of orthohydrogen in the crystal are rotating about as freely as in the gas; ... subsequent to this discovery the phenomenon of rotation of molecules in crystals was found to be not uncommon.
  • Linus Pauling, (1935). "The structure and entropy of ice and of other crystals with some randomness of atomic arrangement". Journal of the American Chemical Society 57 (12): 2680-2684.

• Use "entropy" and you can never lose a debate, von Neumann told Shannon - because no one really knows what "entropy" is.
  • William Poundstone, Fortune's Formula (2005) Part One, "Entropy, Randomness, Disorder, Uncertainty" p. 57.

• My colleague Paul Glansdorff and I have investigated the problem as to if the results of near-equilibrium can be extrapolated to those of far - from-equilibrium situations and have arrived at a surprising conclusion: Contrary to what happens at equilibrium, or near equilibrium, systems far from equilibrium do not conform to any minimum principle that is valid for functions of free energy or entropy production.
  • Ilya Prigogine (1996) "The End of Certainty: Time, Chaos, and the New Laws of Nature". p. 64. Cited in: Ilya Prigogine at echt info. By Sadi-Carnot et all., Jan 28 2013.

• The functional order maintained within living systems seems to defy the Second Law; nonequilibrium thermodynamics describes how such systems come to terms with entropy.
  • Ilya Prigogine, Gregoire Nicolis & Agnes Babloyants, "Thermodynamics of Evolution" Part I, Physics Today (1972) Vol. 25, November. p. 23-28.

• In an isolated system, which cannot exchange energy and matter with the surroundings, this tendency is expressed in terms of a function of the macroscopic state of the system: the entropy.
  • Ilya Prigogine, Gregoire Nicolis & Agnes Babloyants, "Thermodynamics of Evolution" Part II, Physics Today (1972) Vol. 25, December. p. 38-44.

• Entropy is the price of structure.
  • Ilya Prigogine, Isabelle Stengers, Order Out of Chaos: Man's New Dialogue with Nature (1984)

• As far as we know, entropy increases throughout the portion of the universe observable from Earth. It does not seem probable to us, but in any case nothing excludes, that beyond the particle horizon which marks the maximum limit of observations there exist regions in which the arrow of time is reversed compared to ours and in which entropy decreases. I dare not think of the theoretical and observational complications that would arise if the matter contained in one of these anomalous regions began to interact with ours.
  • Tullio Regge, Infinito Viaggio ai limiti dell'universo, Arnoldo Mondadori Editore, Milano, 1995. ISBN 88-04-35976-5. Ch. 8, p. 52

• My greatest concern was what to call it. I thought of calling it 'information,' but the word was overly used, so I decided to call it 'uncertainty.' When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, 'You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage.'
  • Claude Elwood Shannon, Scientific American (1971), Vol. 225, p. 180. Explaining why he named his uncertainty function "entropy"

• Prigogine was also concerned with the broader philosophical issues raised by his work. In the 19th century the discovery of the second law of thermodynamics, with its prediction of a relentless movement of the universe toward a state of maximum entropy, generated a pessimistic attitude about nature and science. Prigogine felt that his discovery of self-organizing systems constituted a more optimistic interpretation of the consequences of thermodynamics. In addition, his work led to a new view of the role of time in the physical sciences.
  • Society for Industrial and Applied Mathematics, Obituaries: Ilya Prigogine SIAM News (September 2003) Vol. 36, No. 7.

• Why is entropy at the beginning of time so low, and the entropy in a black hole so high? ...We ...don't know that the entropy was low ...We don't even know if there was a beginning of time. ...[E]ntropy ...is the physicist's measure of how messy things are, so my room ...tends to get higher and higher entropy, messier and messier. Why... eggs fall on the floor and break, and not... fly up and unbreak? People argued about that for a very long time until the shocking insight... that it was very low 13.4 billion years ago at the time when those... baby pictures of our universe were given off... the cosmic microwave background. ...So somehow, our flow of time towards greater messiness has something to do with our origin of our universe? That... we have learned. ...But now the question of why was that is something where many of my colleagues disagree violently... I have written a paper... which... has very little support... anyway, ...if you take seriously the idea of inflation and also this theory that the wave function does not collapse, according to Hugh Everett, you can do some math and get an explanation... but... it's a wonderful mystery, and I'm open to all ideas... and black holes... is something else we know very little... ultimately where there are great truths yet to be discovered.
  • Max Tegmark, "Max Tegmark & Eric Weinstein: AI, Aliens, Theories of Everything, and New Year’s Resolutions!" (Dec 31, 2020) A YouTube video, 40:06.

• One could... safely declare that 'Physics... can be defined as that subject which treats of the transformation of energy.' The philosophical version of Herakleitos and Empedokles... a continual cycle of changes and exchanges, had... crystallized into a quantitative physical theory. But this... picture... was... incomplete. For... there was a second, equally general and fundamental element in Nature—a directional one. This had first been formulated in the 1820s by the Mozart of modern physics, Sadi Carnot. ...Carnot started with the question: What proportion of the heat in any system is 'available' as a means of producing mechanical energy? ...Carnot demonstrated ...a one-hundred-per-cent-efficient engine could exploit only a fraction of the heat supplied to it... A 'super-efficient' machine which could exploit all the heat supplied, would be (as Carnot's mathematics proved) a perpetual motion machine... one could get out of it more energy than was supplied... In an isolated system... physical changes could at most be perfectly reversible; [but] in normal cases they would result in the progressive... 'degradation' of mechanical energy by the production of unavailable heat. To characterize this... Clausius coined the word 'entropy'... [T]he directional principle of Carnot and Clausias (which gave precise expression to Newton's insight that 'motion is more easily lost than got, and is continually upon the decrease') became the Second Law of Thermodynamics.
  • Stephen Toulmin, June Goodfield, The Architecture of Matter (1962)

• The third model regards mind as an information processing system. This is the model of mind subscribed to by cognitive psychologists and also to some extent by the ego psychologists. Since an acquisition of information entails maximization of negative entropy and complexity, this model of mind assumes mind to be an open system.
  • Thaddus E. Weckowicz, Models of Mental Illness: Systems and Theories of Abnormal Psychology (1984) p. 102.

• It is my thesis that the physical functioning of the living individual and the operation of some of the newer communication machines are precisely parallel in their analogous attempts to control entropy through feedback. Both of them have sensory receptors as one stage in their cycle of operation: that is, in both of them there exists a special apparatus for collecting information from the outer world at low energy levels, and for making it available in the operation of the individual or of the machine. In both cases these external messages are not taken neat, but through the internal transforming powers of the apparatus, whether it be alive or dead. The information is then turned into a new form available for the further stages of performance. In both the animal and the machine this performance is made to be effective on the outer world. In both of them, their performed action on the outer world, and not merely their intended action, is reported back to the central regulatory apparatus. This complex of behavior is ignored by the average man, and in particular does not play the role that it should in our habitual analysis of society; for just as individual physical responses may be seen from this point of view, so may the organic responses of society itself. I do not mean that the sociologist is unaware of the existence and complex nature of communications in society, but until recently he has tended to overlook the extent to which they are the cement which binds its fabric together.
  • Norbert Wiener, Cybernetics and Society (1950) p. 26-27.

• Progress imposes not only new possibilities for the future but new restrictions. It seems almost as if progress itself and our fight against the increase of entropy intrinsically must end in the downhill path from which we are trying to escape.
  • Norbert Wiener, Cybernetics and Society (1950) p. 46-47.

• He sat in the window thinking. Man has a tropism for order. Keys in one pocket, change in another. Mandolins are tuned G D A E. The physical world has a tropism for disorder, entropy. Man against Nature . . . the battle of the centuries. Keys yearn to mix with change. Mandolins strive to get out of tune. Every order has within it the germ of destruction. All order is doomed, yet the battle is worth while.
  • Nathanael West, Miss Lonelyhearts (1933) p.104. Original pages were unnumbered. Page number per the Nathanael West, "Complete Works" Picador Classics paperback, 1988 edition.

• Revolution is everywhere, in everything. It is infinite. There is no final revolution, no final number. The social revolution is only one of an infinite number of numbers: the law of revolution is not a social law, but an immeasurably greater one. It is a cosmic, universal law—like the laws of the conservation of energy and of the dissipation of energy (entropy). Some day, an exact formula for the law of revolution will be established. And in this formula, nations, classes, stars—and books—will be expressed as numerical quantities.
  • Yevgeny Zamyatin, On Literature, Revolution, Entropy and Other Matters (1923) Tr. Mirra Ginsburg, A Soviet Heretic: Essays by Yevgeny Zamyatin (1970) pp. 107-108.

The Origin and Development of the Quantum Theory (June 2, 1920)

by Max Planck

• [M]y previous studies on the second law of thermodynamics served me here... in that my first impulse was to bring not the temperature but the entropy of the resonator into relation with its energy, more accurately not the entropy itself but its second derivative with respect to the energy... [T]his differential coefficient [R]... has a direct physical significance for the irreversibility of the exchange of energy between the resonator and the radiation.

• But as I was... too much devoted to pure phenomenology to inquire more closely into the relation between entropy and probability, I felt compelled to limit myself to the available experimental results. Now, at that time... 1899, interest was centred on the law of the distribution of energy... proposed by W. Wien... On calculating the relation following from this law between the entropy and energy of a resonator the remarkable result is obtained that the reciprocal value of the above differential coeffcient... R, is proportional to the energy. This extremely simple relation can be regarded as an adequate expression of Wien's law...

• I was... occupied with the task of giving it a real physical meaning, and this... led me, along Boltzmann's line... to the consideration of the relation between entropy and probability... after some weeks of the most intense work of my life clearness began to dawn... and an unexpected view revealed itself...

• Entropy, according to Boltzmann, is a measure of a physical probability, and the meaning of the second law of thermodynamics is that the more probable a state is, the more frequently will it occur in nature.

• [W]hat one measures are only the differences of entropy, and never entropy itself, and consequently one cannot speak... of the absolute entropy of a state. But nevertheless the introduction of an appropriately defined absolute magnitude of entropy is... recommended... by its help certain general laws can be formulated with great simplicity.

• The significant part played in the origin of the classical thermodynamics by mental experiments is now taken over in the quantum theory by P. Ehrenfest's hypothesis of the adiabatic invariance; and just as the principle introduced by R. Clausius, that any two states of a material system are mutually interconvertible on suitable treatment by reversible processes, formed the basis for the measurement of entropy, just so do the new ideas of Bohr show a way into the midst of the wonderland he has discovered.

The Nature of the Physical World (1928)

by Arthur Eddington

• Entropy continually increases. We can, by isolating parts of the world and postulating rather idealized conditions... arrest the increase, but we cannot turn it into a decrease. ...The law that entropy always increases—the second law of thermodynamics—holds... the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is... found to be against the second law... I can give you no hope; there is nothing for it but to collapse in deepest humiliation.
  • p. 74.

• I wish I could convey to you the amazing power of this conception of entropy in scientific research. From the property that entropy must always increase, practical methods of measuring it have been found. The chain of deductions from this simple law have been almost illimitable... equally successful in... theoretical physics and the practical tasks of the engineer. ...It is not concerned with the nature of the individual; it is interested in him only as a component of the crowd. ...[T]he method is applicable in fields of research where our ignorance has scarcely begun to lift ...
  • p. 75.

• Thermodynamical Equilibrium. Progress of time introduces more of the random element into the constitution of the world. ...[T]he world contains both chance and design, or... antithesis of chance. ...[O]ur method of measurement of entropy: we assign to the organization or non-chance element a measure... proportional to the strength of our disbelief in a chance origin of it. ...The scientific name for a fortuitous concourse of atoms is "thermodynamic equilibrium". ...Thermodynamic equilibrium is the... case... in which no increase in the random element can occur... [i.e.] shuffling is... as thorough as possible. ...In such a region we lose time's arrow. ...[T]he arrow points in the direction of increase of the random element. ...The arrow does not know which direction to point.
  • pp. 77-79.

• Is the random element... the only feature of the physical world which can furnish time with an arrow? ...Nothing in the statistics of an assemblage can distinguish a direction of time when entropy fails to distinguish one. ...[T]his law was only discovered in the last few years ...It is accepted as fundamental in ...atoms and radiation and had proved to be one of the most powerful weapons of progress in such researches. It does not seem to be... deducible from the second law...
  • p. 79.

• Whilst the physicist would... say that the matter of this... [dining] table... is really a curvature of space, and its colour is really an electromagnetic wavelength, I do not think that he would say that the familiar moving on of time is really an entropy-gradient. ...[T]here is something as yet ungrasped behind the notion of entropy—some mystic interpretation... not apparent in the definition... [W]e strive to see that entropy-gradient may really be the moving on of time (instead of vice-versa).
  • p. 95.

• The more closely we examine the association of entropy with "becoming" the greater do the obstacles appear. If entropy were one of the elementary indefinables of physics there would be no difficulty. Or if the moving on of time were something of which we were made aware through our sense organs there would be no difficulty. ...Suppose that we had to identify "becoming" with an electrical potential-gradient ...through the readings of a voltmeter.
  • p. 96.

• [S]uppose that we had to identify force with entropy-gradient. That would only mean that entropy-gradient is a condition which stimulates a nerve, which thereupon transmits an impulse to the brain, out of which the mind weaves its own peculiar impression of force. ...It is absurd to pretend that we are in ignorance of the nature of organisation in the external world in the same way that we are ignorant of the intrinsic nature of potential. It is absurd to pretend that we have no justifiable conception of "becoming"... That dynamical quality... has to do much more than pull the trigger of a nerve. ...a moving on of time is a condition of consciousness. ...It is the innermost Ego of all which is and becomes.
  • p. 97.

• Consciousness, besides detecting time's arrow, also roughly measures the passage of time. ...but is a bit of a bungler in carrying it out. ...Our consciousness somehow manages to keep ...record of the flight of time ...reading some kind of clock in the material brain ...a better analogy would be an entropy-clock ...primarily for measuring the rate of disorganisation of energy, and only roughly keeping pace with time. ...[I]n forming our ideas of duration and of becoming... [e]ntropy-gradient is... the direct equivalent of the time of consciousness in both... aspects. Duration measured by physical clocks (time-like interval) is only remotely connected.
  • pp. 100-101.

• [T]he conception associated with entropy... marked a reaction from the view that everything to which science must pay attention is discovered by a microscopic dissection of objects. ...[T]he centre of interest is shifted from the entities reached by the customary analysis (atoms, electric potentials, etc.) to qualities possessed by the system as a whole... The artist... resorts to an impressionist painting. ...[T]he physicist has found ...his impressionist scheme is just as much exact science and even more practical ...than his microscopic scheme.
  • p. 103.

• Entropy... was discovered and exalted because it was essential to practical applications of physics... But by it science has been saved from a fatal narrowness. ...[T]here would have been nothing to represent "becoming" in the physical world.
  • p. 104.

• Entropy was not in the same category as the other physical quantities ...and the extension ...was in a very dangerous direction. ...But entropy had secured a firm place in physics before it was discovered that it was a measure of the random element in arrangement. It was in great favour with the engineers. ...[A]t that time it was the general assumption that the Creation was the work of an engineer (not of a mathematician, as is the fashion nowadays).
  • p. 104.

• Suppose that we are asked to arrange the following in two categories—
  distance, mass, electric force, entropy, beauty, melody.
  [T]here are the strongest grounds for placing entropy alongside beauty and melody... Entropy is only found when the parts are viewed in association... [as are] beauty and melody. All three are features of arrangement. ...The reason why this [entropy] stranger can pass itself off among the aborigines of the physical world is... the language of arithmetic. It has... measure-number... at home in physics.
  • p. 105.

• It had become the regular outlook of science... that constellations are not to be taken seriously, until the constellation of entropy made a solitary exception. When we analyze the picture into a large number of particles of paints, we lose the aesthetic significance of the picture. The particles... go into the scientific inventory, and it is claimed that everything that there really was in the picture is kept. But this way of keeping... may be... losing ... The essence of a picture... is arrangement.
  • p. 106.

• I cannot read any significance into a physical world that is held... upside down. For that reason I am interested in entropy not only because it shortens calculations which can be made by other methods, but because it determines an orientation which cannot be found by other methods. ...[T]ime makes a dual entry and thus forms an intermediate link between the internal and the external. This is shadowed partially by the scientific world of primary physics (which excludes time's arrow), but fully when we... include entropy. ...[It] has generally been assumed that the object of the quest is to find out all that really exists. There is another quest... to find out all that really becomes.
  • p. 109.

• The discrimination between cause and effect depends on time's arrow and can only be settled by reference to entropy.
  • p. 110.

• Except for action and entropy (which belongs to an entirely different class of physical conceptions) all the quantities prominent in pre-relativity physics refer to the three-dimensional sections which are different for different observers.
  • pp. 180-181.

• I am standing on the threshold about to enter a room. ...I must make sure of landing on a plank travelling twenty miles a second around the sun—a fraction of a second too early or too late, the plank would be miles away. I must do this whilst hanging from a round planet head outward into space, and with a wind of aether... I ought really to look at the problem four-dimensionally as concerning the intersection of my world-line with that of the plank. Then again it is necessary to determine in which direction the entropy of the world is increasing in order to make sure that my passage over the threshold is an entrance, not an exit.
  Verily, it is easier for a camel to pass through the eye of a needle than for a scientific man to pass through a door. And whether... barn... or church door it might be wiser that he should consent to be an ordinary man... rather than wait til all the difficulties in... scientific ingress are resolved.
  • p. 342.

What Is Life? (1944)

The Physical Aspect of the Living Cell by Erwin Schrödinger. A source (1945). Page numbers refer to 1945 edition, unless otherwise noted.

• When is a piece of matter said to be alive? When it goes on... moving, exchanging material with its environment... When a system... is not alive... all motion usually comes to a standstill... as a result of friction... [T]he whole system fades away into a dead, inert lump of matter. A permanent state is reached, in which no observable events occur. The physicist calls this the state of thermodynamic equilibrium, or of 'maxiumum entropy'.
  • p. 70.

• What is entropy? ...a measurable physical quantity just like the length ...temperature ...the heat of fusion ...or the specific heat of any given substance. At ...absolute zero ...the entropy of any substance is zero. When you bring the substance into any other state by slow, reversible little steps ...the entropy increases by an amount computed by dividing every little portion of heat you had to supply ...by the absolute temperature at which it was supplied ...and by summing up all these small contributions.
  • pp. 72-73.

• [T]he statistical concept of order and disorder... was revealed by... Boltzmann and Gibbs... This too is an exact quantitative connection...entropy = ${\displaystyle k\log D}$where ${\displaystyle k}$ is the... Boltzmann constant and ${\displaystyle D}$ is... the atomistic disorder of the body... The disorder... is partly... heat of motion, partly... atoms and molecules being mixed at random... e.g., sugar and water molecules... The gradual 'spreading out' of the sugar over all the water... increases the disorder ${\displaystyle D}$, and hence (since the logarithm of ${\displaystyle D}$ increases with ${\displaystyle D}$) the entropy. ...[A]ny supply of heat increases the turmoil of heat motion, that is ...increases ${\displaystyle D}$... [W]hen you melt a crystal... you... destroy the neat and permanent arrangement of... atoms or molecules and turn the crystal lattice into a continually changing random distribution.
  • p. 73.

• If ${\displaystyle D}$ is a measure of disorder... ${\displaystyle 1/D}$... can be regarded as a... measure of order. Since the logarithm of ${\displaystyle 1/D}$ is... minus the logarithm of ${\displaystyle D}$...-(entropy) = ${\displaystyle k\log 1/D}$
  • p. 74.

• [T]he device by which an organism maintains itself stationary at a fairly high level of orderliness (...low level of entropy) ...consists in continually sucking orderliness from its environment.
  • p. 74.

• [H]igher animals... feed upon... the extremely well-ordered state of... foodstuffs. After utilizing it they return it in a... degraded form—not entirely degraded... for plants can... use... it. (...[Plants] have their most powerful supply of 'negative entropy' in the sunlight).
  • p. 75.

• The remarks on negative entropy have met with doubt and opposition from physicist colleagues. ...[I]f I had been catering for them alone I should have let the discussion turn on free energy instead. It is the more familiar notion... [b]ut seemed linguistically too near energy for... the average reader... the concept is a rather intricate one, whose relation to Boltzmann's order-disorder principle is less easy to trace... '[E]ntropy with a negative sign'... is not my invention. It... [is] precisely the thing on which Boltzmann's original argument turned.
  • p. 83. Notes to section VI (1956)

• Energy is needed to replace not only the mechanical energy of our bodily exertions, but also the heat we continually give off... And that we give off heat is not accidental, but essential. For this is precisely the manner in which we dispose of the surplus entropy we continually produce in our... life process.
  • p. 84. Notes to section VI (1956)

• Nernst's discovery was induced by the fact that even at room temperature entropy plays an astonishingly insignificant role in many chemical reactions.
  • p. 85.

See also

• Ludwig Boltzmann
• Lazare Carnot
• Nicolas Léonard Sadi Carnot
• Rudolf Clausius
• Energy
• Josiah Willard Gibbs
• Information theory
• James Clerk Maxwell
• Thermodynamics
  • Second Law of Thermodynamics/Entropy

External links

• Encyclopedic article on Entropy on Wikipedia
• The dictionary definition of entropy on Wiktionary

• Entropy from Sixty Symbols, University of Nottingham
• Lecture 9: Entropy and the Clausius inequality, Thermodynamics & Kinetics, MIT OpenCourseWare (Spring 2008, Undergraduate)
• Lecture 24 - The Second Law of Thermodynamics (cont.) and Entropy, PHYS 200: Fundamentals of Physics I, Open Yale Courses, Yale University

• YouTube Videos
  • Adam Shulman on "The Discovery of Entropy" from the "Program on Constitutional Government at Harvard" channel
  • Entropy and the Second Law of Thermodynamics from the DrPhysicsA channel
  • Khan Academy, from their Chemistry Playlist
    • Entropy Intuition
    • More on Entropy
    • Proof: S (or Entropy) is a valid state variable
    • Reconciling Thermodynamic and State Definitions of Entropy
    • Thermodynamic Entropy Definition Clarification