Talk is tough when hands are tied

Eighteen-month-old Timothy gestured towards the circus and said: "Da!" (Quite his commonest word, with several meanings.) "Yes, Timmy, there's the circus," I said, to which five-year old Franziska added: "Daddy, he's not just pointing at it, he really wants to go to it." There, in a nutshell, are the problems of gesture. From an early age, it is used to communicate, it can be read by a five-year-old and its interpretation is laden with ambiguities.

Gestures start early in life, occurring spontaneously even in the congenitally blind. Deaf signers, their hands already occupied, instead make mouth gestures. Gesture occurs even in those born without arms, as in Mirabelle, who described her phantom arms: "As I am talking to you, they are gesticulating... In fact, they're moving now as I speak."

In Hearing Gesture , Susan Goldin-Meadow summarises 20 years of her own and others' research. She comments on topics that range from the gestural communication systems used by workers in sawmills, where noise prevents talk, and Trappist monks, whose vows preclude talk, to Australian aboriginal women silent while in mourning, and the gestural lingua franca of Plains Indians.

Not all hand movements are gesture in a technical sense. The infamous two fingers or a cheery thumbs-up are "emblems", while simple nervous habits such as smoothing hair or beard-stroking are "adaptors" - they are not tied to speech temporally or semantically. What Goldin-Meadow means by gesture or gesticulation is shown by someone saying: "Immediately, I ran up the stairs to get the book." The type of staircase is shown by a finger spiralling up, the movements time-locked to the speech, starting on "ran" and finishing on "stairs". Say: "Slowly, I walked up the (non-spiral) stairs", and the gestures are different.

But why is gesture needed? Language efficiently conveys information, so what does gesture add? Would hearing me read these words be different from watching a video of me with arms flailing? Hearing Gesture addresses these and other questions, and for the next decade will be the starting point for those interested in gesture.

At times it is a leaden read, with too many experimental details, histograms and hypotheses that confuse the narrative. Part of the problem is using words and the occasional picture to describe gestures. Just as a newspaper report of a football match cannot compare with a televised game, so I yearned for a CD-Rom or website showing the subtleties of the video clips that the text struggled to describe.

Many of the experiments describe children using a whiteboard to explain their attempts to find the missing number in equations such as 7+6+5=___+15. In maths, gesture has several roles:it can indicate which number is being referred to or can be a "placeholder" to indicate one's place in a calculation. Gesture, though, is also a genuinely spatial form of communication, ideally suited in maths for dynamically representing locations and movements in three dimensions, which the sequential, linear symbols of speech can never do. Just try describing the parabolic trajectory of a cannon ball verbally without moving your hands.

Much of the first part of the book focuses on what Goldin-Meadow calls "mismatches", a less-than-happy term in which "gesture and speech convey different information". Mismatch, she emphasises, does not imply "the notion of conflict". Instead, parallel channels communicate different types of information (as pictures and words complement and support one another on the printed page).

Mismatches occur, for instance, when a child describes one part of an equation but points to another. Goldin-Meadow suggests, not always compellingly, that mismatches indicate a readiness to learn, the hand, as it were, reaching out for alternative problem solutions. A strong practical point is that teachers can be trained to notice mismatches that indicate a student is "open to instruction and thus on the precipice of learning", so that effective intervention can be made.

The latter part of the book is more satisfactory, asking fundamental questions about the nature of gesture and emphasising that it is no mere epiphenomenon but integral to speech perception and production, and to cognition itself. At first sight, simple experiments suggest gesture is unimportant in language perception, speech alone being understood as well as speech with video. Here, though, mismatches play a more compelling role.

Actors using inappropriate hand gestures are misunderstood more often than when they use appropriate gestures. Gesture helps communication, but does it benefit the communicator? Gesture increases as communication becomes difficult, as with aphasia or delayed auditory feedback or when speaking a second language, and acts to reduce the speaker's cognitive load.

As a demonstration, sit alone in a quiet room and explain something out loud. Now do it while sitting on your hands. It feels more difficult, as anyone knows who has been told not to wave their arms in a TV studio. Does gesture change the content of speech, though? Fascinating experiments describe how preventing gesturing reduces the vividness of imagery, and making subjects rotate their hands in a circle while telling a story produces more cyclic imagery, whereas side-to-side movements generate more linear allusions.

Goldin-Meadow makes a strong case for tight integration of gesture with normal speech. What is missing, though, is a sense of the origins of this parallel, partly independent communication system. Are these separate actions of left and right hemispheres perhaps, and can brain scans inform us about gesture perception and generation in relation to speech? More crucial, is gesture perhaps only an atavistic residue of a primitive, pre-linguistic communication system used for hunting on the African savannah? Or did it somehow drive language development, or evolve hand-in-hand with spoken language? Indeed, is gesture neurologically restricted to the hands and, if it is, what is the role of gestural movements in ballet and dance?

Chris McManus is professor of psychology and medical education, University College London.