Searching for causes and codes of complexity that allow squirrels to fly to the 'adjacent possible'

At one point in his daunting, sometimes exhilarating new book, Investigations , Stuart Kauffman ponders whose genius is to be more coveted: Einstein's or Shakespeare's? This offhand reverie occurs in a wide-ranging work long on theory and short on natural history or facts. Its goal at times seems as much to dazzle and bamboozle us with his brilliance as to inform us. What are we to make of this theorist who has emerged from the edge of cultural chaos to become the premier spokesman of complexity, courted by corporations and politicians, an apprentice jazz drummer with the ear of a handful of Nobel prizewinners, the chief and most charming interdisciplinarian guru of the most exciting abstract think-tank in the world, the Santa Fe Institute?

Although Kauffman may be buying into his own hype, it is difficult to resist the insistence of his vision. At his best, he selects from among the secret processes of his mathematical eclecticism some glittering new jewel for our covetous inspection, some abstract gem. His theory that the biosphere is racing "as fast as possible" into "the adjacent possible", for example, intriguingly highlights the fact that the time elapsed since the origin of the universe has been woefully inadequate to accommodate all possible combinations of its constituent particles. The biosphere is, in mathematicians' terms, "non-ergodic", ie non-repeating. This is important because thermodynamics - the study of energy and its transformations - has in modern times been based on the statistics of mixing. When cream mixes with coffee, they come to equilibrium: there are far more ways for the constituents to be mixed than separated. So in the forward flow of time the constituents naturally intermingle, and the temperature of the coffee equilibrates with that of the room. Such mixing was even thought by Boltzmann, the founder of statistical mechanics, to provide the direction of linear time. But Poincare and others showed that the direction of increasing probability does not ensure mixing. Over infinite time, even statistically rare combinations would occur, and not once but an infinite number of times. Einstein and Godel subsequently worked on trying to find a better reason for time's apparent one-way flow, but they failed.

Indeed, even the equations of Einstein and Bohr and others - which underlie virtually all of modern science - are flawed in their assumption that we can state in advance the environment we want to predict. We cannot just apply laws because evolution is too strong. Drawing (with some reservations) on the high-concept work of cosmologist and friend Lee Smolin, who argues for a cosmic selection in which universal constants are born in black holes, Kauffman says what we call laws are not given but must occur over evolutionary time, like human legal codes. The regime of chromosomes and meiosis in cells, for example, operates as a law even though it evolved in time. A trilobite that jumped left, instead of right, and was devoured, took the genes of an entire lineage with her, never to be seen again. The first flying squirrel really just had ugly flaps of skin that came in handy when she jumped. Such evolutionary events, occurring as the cosmos (and with it life) heads non-ergodically into the "adjacent possible", cannot be finitely pre-stated, Kauffman tells us, over and again. Indeed, this is the best part of his book - though ultimately it may be Kauffman's unconstrained mathematical pyrotechnics, not more slow-moving reality, that expands most quickly into the adjacent possible. As a philosophy, the adjacent possible may seem reminiscent of Milan Kundera's viewpoint in The Unbearable Lightness of Being , though it has the heaviness of Nietzsche in his ergodic view of eternal recurrence, where everything that has happened or is chosen to happen will and must happen, again and again.

But there is a big problem with the science, especially Kauffman's ambitious attempt to derive a new, fourth law of thermodynamics. To the person with a hammer, everything is a potential nail: to the complexity theorist, everything looks like algorithmic deterministic complexity, everything is similar to a program on a personal computer. Kauffman recognises this weakness. His new "constructivist" (rather than reductionist) science is implicitly a critique of algorithmic complexity, of the computation of outcomes based on initial conditions made so much easier by the PC. His Boolean algebraic explanation of how regulatory genes connecting cells can narrow a huge number into a manageable one ("order for free") agrees with the actual quantity of human structural genes and the related number of human cell types.

There are simpler, better explanations of life's defiance of algorithmic complexity than a law-like flight of life, the universe and everything into the adjacent possible. Not coincidentally, these explanations come from thermodynamics - but a biological thermodynamics Kauffman has neglected. First of all, if the universe were collapsing as fast as possible into the adjacent possible, becoming as complex as it can, it might be teeming with life rather than radiation and matter. Better than to devise a new, fourth law (and then retrofitting it to the cosmos) is to extend the second law. It is strange that a book on biospheric complexity does not mention Vladimir Vernadsky or Alfred Lotka. It is crucial to remember that the second law was originally stated for isolated systems rather than the open ones of life and the cosmos; that its original incarnation thus covered the special rather than the general case. This is why it must be extended, as indeed it has been in an anthology called What is Life : The Next Fifty Years (to which Kauffman contributed), by the thermodynamicist Eric D. Schneider. Dramatically contrasting with Kauffman's complex charts and explanations, Schneider elegantly writes that "nature abhors a gradient".

This simple, impersonal statement is quantitatively if unwittingly applied in Eric Chaisson's Cosmic Evolution : The Rise of Complexity in Nature . Chaisson, unlike Kauffman, grounds his mathematics in actual measurement of energy flow in complex systems; and while he also ranges far and wide, explaining the rise of complexity in a cosmic context, he explicitly eschews the need for a new law, depending instead on a novel application of non-equilibrium thermodynamics to his deep grasp of astrophysical facts. The result is a fascinating new synthesis of how energy flow has acted to bring about complexity, especially in the pre-life universe.

Using a measure he calls the "free energy density rate", basically a proxy for energy flow, Chaisson shows how it measurably increases in complex systems from galaxies through stars to biospheres, reptiles, mammals, brains, societies and computers. Although it is not perfect (it is anomalously high in flames, hummingbirds and respiring bacteria), the free energy density rate shows the general direction of complexity and underscores the crucial point that evolving complexity depends on energy flow through systems. The measure is also hampered because fast-growing systems (eg children, early-stage ecosystems) come out with higher values than their more diverse, adult stages.

But this is to be expected with any single number that measures a developing system with a snapshot. Chaisson shows how gravitation (which Kauffman explicitly ignores) brought the cosmos, some 100,000 years after the big bang, out of thermodynamic equilibrium as low-entropy stars formed, producing temperature and other gradients and setting up the stage for the evolution of future complexity, including life, which reduces the solar gradient in perfect accord with the second law. Ultimately, in Chaisson's view, it is the expansion of space that drives complexity; stars do not make the sky white at night because they are shining into cold space as part of a huge gradient. Chaisson misses the point that complex systems not only do not violate the second law (albeit an important point) but that their complexity, producing concomitant disorder (molecular chaos), reduces gradients more efficiently than would otherwise be the case. A hurricane, for example, reduces a pressure gradient: the spinning of air does a job, has an organisation and a complexity that is not the result of itself, but of the gradients around it. Chaisson also (though he has read and references the biological thermodynamicists Lotka, Jeffrey Wicken, and Schneider) misses that the increase of species over time represents new cyclical (non-ergodic) pathways for gradient breakdown - as well as the fact that ecological succession, where energy plays an obvious ordering role, can be regarded as a microcosm of the thermodynamic evolution of complexity. Nonetheless, Chaisson's book provides exciting new testimony to the increasing power of non-equilibrium thermodynamics to change how we see ourselves and the world.

Lynn Margulis is professor in the department of geosciences, University of Massachusetts, Amherst, United States. Dorion Sagan is a science writer.