Entropy: The new order
- 27 August 2005
- From New Scientist Print Edition. Subscribe and get 4 free issues.
- Mark Buchanan
CONSTANTINO TSALLIS has a single equation written on the blackboard in his office. It looks like one of the most famous equations in physics, but look more closely and it's a little bit different, decorated with some extra symbols and warped into a peculiar new form.
Tsallis, based at the Brazilian Centre for Research in Physics, Rio de Janeiro, is excited to have created this new equation. And no wonder: his unassuming arrangement of symbols has stimulated hundreds of researchers to publish more than a thousand papers in the past decade, describing strange patterns in fluid flows, in magnetic fields issuing from the sun and in the subatomic debris created in particle accelerators. But there is something even more remarkable about Tsallis's equation: it came to him in a daydream.
In 1985, in a classroom in Mexico City, Tsallis was listening as a colleague explained something to a student. On the chalkboard they had written a very ordinary algebraic expression, pq, representing some number p raised to the power q In Tsallis's line of work - describing the collective properties of large numbers of particles - the letter "p" usually stands for probability: the probability that a particle will have a particular velocity, say. Tsallis stared at the formula from a distance and his mind drifted off. "There were these pqs all over the board," he recalls, "and it suddenly came to my mind - like a flash - that with powers of probabilities one might do some unusual but possibly quite interesting physics."
The physics involved may be more than quite interesting, however. The standard means of describing the collective properties of large numbers of particles - known as statistical mechanics - has been hugely successful for more than a century, but it has also been rather limited in its scope: you can only apply it to a narrow range of systems. Now, with an insight plucked out of thin air, Tsallis may have changed all that.
Developed in the 19th century, statistical mechanics enabled physicists to overcome an imposing problem. Ordinary materials such as water, iron or glass are made of myriad atoms. But since it is impossible to calculate in perfect detail how every individual atom or molecule will move, it seems as if it might never be possible to understand the behaviour of such substances at the atomic level.
The solution, as first suggested by the Austrian physicist Ludwig Boltzmann, lay in giving up hope of perfect understanding and working with probabilities instead. Boltzmann argued that knowing the probabilities for the particles to be in any of their various possible configurations would enable someone to work out the overall properties of the system. Going one step further, he also made a bold and insightful guess about these probabilities - that any of the many conceivable configurations for the particles would be equally probable.
Deeper beauty
Boltzmann's idea works, and has enabled physicists to make mathematical models of thousands of real materials, from simple crystals to superconductors. But his work also has a deeper beauty. For a start, it reflects the fact that many quantities in nature - such as the velocities of molecules in a gas - follow "normal" statistics. That is, they are closely grouped around the average value, with a "bell curve" distribution.
The theory also explains why, if left to their own devices, systems tend to get disorganised. Boltzmann argued that any system that can be in several different configurations is most likely to be in the more spread out and disorganised condition. Air molecules in a box, for example, can gather neatly in a corner, but are more likely to fill the space evenly. That's because there are overwhelmingly more arrangements of the particles that will produce the spread out, jumbled state than arrangements that will concentrate the molecules in a corner. This way of thinking led to the famous notion of entropy - a measure of the amount of disorder in a system. In its most elegant formulation, Boltzmann's statistical mechanics, which was later developed mathematically by the American physicist Josiah Willard Gibbs, asserts that, under many conditions, a physical system will act so as to maximise its entropy.
And yet Boltzmann and Gibbs's statistical mechanics doesn't explain everything: a great swathe of nature eludes its grasp entirely. Boltzmann's guess about equal probabilities only works for systems that have settled down to equilibrium, enjoying, for example, the same temperature throughout. The theory fails in any system where destabilising external sources of energy are at work, such as the haphazard motion of turbulent fluids or the fluctuating energies of cosmic rays. These systems don't follow normal statistics, but another pattern instead.
”In Mexico City, coming out of his reverie, Tsallis wrote up some notes on his idea
If you repeatedly measure the difference in fluid velocity at two distinct points in a turbulent fluid, for instance, the probability of finding a particular velocity difference is roughly proportional to the amount of that difference raised to the power of some exponent. This pattern is known as a "power law", and such patterns turn up in many other areas of physics, from the distribution of energies of cosmic rays to the fluctuations of river levels or wind speeds over a desert. Because ordinary statistical mechanics doesn't explain power laws, their atomic-level origins remain largely mysterious, which is why many physicists find Tsallis's mathematics so enticing.
In Mexico City, coming out of his reverie, Tsallis wrote up some notes on his idea, and soon came to a formula that looked something like the standard formula for the Boltzmann-Gibbs entropy - but with a subtle difference. If he set q to 1 in the formula - so that pq became the probability p - the new formula reduced to the old one. But if q was not equal to 1, it made the formula produce something else. This led Tsallis to a new definition of entropy. He called it q entropy.
Back then, Tsallis had no idea what q might actually signify, but the way this new entropy worked mathematically suggested he might be on to something. In particular, the power-law pattern tumbles out of the theory quite naturally. Over the past decade, researchers have shown that Tsallis's mathematics seem to describe power-law behaviour accurately in a wide range of phenomena, from fluid turbulence to the debris created in the collisions of high-energy particles. But while the idea of maximising q entropy seems to work empirically, allowing people to fit their data to power-law curves and come up with a value of q for individual systems, it has also landed Tsallis in some hot water. The new mathematics seems to work, yet no one knows what the q entropy really represents, or why any physical system should maximise it.
Degrees of chaos
And for this reason, many physicists remain sceptical, or worse. "I have to say that I don't buy it at all," says physicist Cosma Shalizi of the University of Michigan in Ann Arbor, who criticises the mathematical foundations of Tsallis's approach. As he points out, the usual Boltzmann procedure for maximising the entropy in statistical mechanics assumes a fixed value for the average energy of a system, a natural idea. But to make things work out within the Tsallis framework, researchers have to fix the value of another quantity - a "generalised" energy - that has no clear physical interpretation. "I have yet to encounter anyone," says Shalizi, "who can explain why this should be natural."
To his critics, Tsallis's success is little more than sleight of hand: the equation may simply provide a convenient way to generate power laws, which researchers can then fit to data by choosing the right value of q "My impression," says Guido Caldarelli of La Sapienza University in Rome, "is that the method really just fits data by adjusting a parameter. I'm not yet convinced there's new physics here." Physicist Peter Grassberger of the University of Wuppertal in Germany goes further. "It is all nonsense," he says. "It has led to no new predictions, nor is it based on rational arguments."
The problem is that most work applying Tsallis's ideas has simply chosen a value of q to make the theory fit empirical data, without tying q to the real dynamics of the system in any deeper way: there's nothing to show why these dynamics depart from Boltzmann's picture of equal probabilities. Tsallis, who is now at the Santa Fe Institute in New Mexico, acknowledges this is a limitation, but suggests that a more fundamental explanation is already on its way.
”Tsallis has reawakened physicists to fundamental questions they have never quite answered
Power laws, he argues, should tend to arise in "weakly chaotic" systems. In this kind of system, small perturbations might not be enough to alter the arrangement of molecules. As a result, the system won't "explore" all possible configurations over time. In a properly chaotic system, on the other hand, even tiny perturbations will keep sending the system into new configurations, allowing it to explore all its states as required for Boltzmann statistics.
Tsallis argues that if physicists can adequately understand the details of this "exploring behaviour", they should be able to predict values of q from first principles. In particular, he proposes, some as yet unknown single parameter - closely akin to q - should describe the degree of chaos in any system. Working out its value by studying a system's basic dynamics would then let physicists predict the value of q that then emerges in its statistics.
Other theoretical work seems to support this possibility. For example, in a paper soon to appear in Physical Review E, physicist Alberto Robledo of the National Autonomous University of Mexico in Mexico City has examined several classic models that physicists have used to explore the phenomenon of chaos. What makes these models useful is that they can be tuned to be more or less chaotic - and so used to explore the transition from one kind of behaviour to another. Using these model systems, Robledo has been able to carry out Tsallis's prescription, deriving a value of q just from studying the system's fundamental dynamics. That value of q then reproduces intricate power-law properties for these systems at the threshold of chaos. "This work shows that q can be deduced from first principles," Tsallis says.
While Robledo has tackled theoretical issues, other researchers have made the same point with real observations. In a paper just published, Leonard Burlaga and Adolfo Vinas at NASA's Goddard Space Flight Center in Greenbelt, Maryland, study fluctuations in the properties of the solar wind - the stream of charged particles that flows outward from the sun - and show that they conform to Tsallis's ideas. They have found that the dynamics of the solar wind, as seen in changes in its velocity and magnetic field strength, display weak chaos of the type envisioned by Tsallis. Burlaga and Vinas have also found that the fluctuations of the magnetic field follow power laws that fit Tsallis's framework with q set to 1.75 (Physica A, vol 356, p 275).
The chance that a more comprehensive formulation of Tsallis's q entropy might eventually be found intrigues physicist Ezequiel Cohen of the Rockefeller University in New York City. "I think a good part of the establishment takes an unfair position towards Tsallis's work," he says. "The critique that all he does is 'curve fitting' is, in my opinion, misplaced."
Cohen has also started building his own work on Tsallis's foundations. Two years ago, with Christian Beck of Queen Mary, University of London, he proposed an idea known as "superstatistics" that would incorporate the statistics of both Boltzmann and Tsallis within a larger framework.
In this work they revisited the limitation of Boltzmann's statistical mechanics. Boltzmann's models cannot cope with any system in which external forces churn up differences such as variations in temperature. A particle moving through such a system would experience many temperatures for short periods and its fluctuations would reflect an average of the ordinary Boltzmann statistics for all those different temperatures. Cohen and Beck showed that such averaged statistics, emerging out of the messy non-uniformity of real systems, take the very same form as Tsallis statistics, and lead to power laws. In one striking example, Beck showed how the distribution of the energies of cosmic rays could emerge from random fluctuations in the temperature of the hot matter where they were originally created.
Cohen thinks that, if nothing else, Tsallis's powers of probabilities have served to reawaken physicists to fundamental questions they have never quite answered. After all Boltzmann's idea, though successful, was also based on a guess; Albert Einstein disliked Boltzmann's arbitrary assumption of "equal probabilities" and insisted that a proper theory of matter had to rest on a deep understanding of the real dynamics of particles.
That understanding still eludes us, but Tsallis may have taken us closer. It is possible that, in his mysterious q entropy, Tsallis has discovered a kind of entropy just as useful as Boltzmann's and especially suited to the real-world systems in which the traditional theory fails. "Tsallis made the first attempt to go beyond Boltzmann," says Cohen. The door is now open for others to follow.
