The Relationship Between Gesture And Thinking/speaking

The relationship between gesture and thinking/speaking

It’s easy to see the relationship between speech and gesture; people move their hands while they speak. There is also a more fundamental relationship between gestures and thinking, a relationship that is possibly even more primary for gesture than that with speech. Earlier this year a Sotaro Kita, Martha W. Alibali and Mingyuan Chu published a paper that brings together a lot of excellent research to discuss the role that gesture has in thinking and speaking. As Kita tweeted when it was was published in January, this paper brings together 20 years of research and 10 years of writing. The authors draw on an extensive background literature, which includes their own extensive and excellent research (you may remember Kita’s work from the ‘Blind people gesture’ post, and Alibali’s work featured in an earlier post about unhelpful gesturing).

If you only read one survey paper on the relationship between gesture and language, this is certainly a good one. But reading papers about gesture is my day job, not yours, so I’ve summarised some of the good bits for you.

Introducing the ‘Gesture-for-Conceptualisation’ model

In this paper the authors argue that the main function of gesture is to help the speaker conceptualise what they are thinking about, and therefore going to speak about. The authors argue that gestures are “physical actions of a special type” (p. 1), which means that gesturing about moving a mug of coffee is linked to the action of moving coffee as much as it is talking about moving the coffee, and that makes gestures particularly important in conceptualising tasks, and also talking about them.

They argues that there are four main ways gesture affects cognition to help conceputalise thought: (1) activation, (2) manipulation, (3) packaging and (4) exploration. All may occur in the one gesture, or one function may be more dominant at a particular time. I’ll go through each of these briefly. I often talk about the relationship between language and gesture being very close, so at the end I’ll also look at what the authors say about why gesture is at an earlier stage of cognition than spoken language might be. 

Gesture activates cognition: Gesturing while thinking silently

The authors present evidence from experiments that show people gesture when thinking about problems, even if they are not speaking and are by themselves, and that these gestures can help people perform better certain kinds of mental tasks that involve thinking about the manipulation of real or imaginary objects. Being allowed to gesture means people in experiments tend to think out solutions to problems using more space and movement, people who aren’t allowed to gesture tend to use more abstract ways of talking through the problem. This means that gesture plays a more central role in cognition than just being linked to one other cognitive skill, like language or problem-solving, which is how it has been discussed in earlier work.

The authors argue that gesture can help with cognition because it represents information about objects and their manipulation in a more “schematic” way than other cognitive processes, stripping out a lot of potentially unnecessary detail.

Gesture manipulates cognition: People gesture more while dealing with complex spatial/motion information

Not only do people gesture more in these tasks, but in one set of experiments people who performed more gestures do better in later attempts at the task, even when they’re told not to gesture on the second attempt - indicating there is some ongoing benefit to cognition, manipulating their cognitive skills at the task. 

Gesture packages cognition: Changing the way people gesture changes their language

Gesture can package information about space and movement. In one experiment researchers used this to manipulate how people used language to discuss an event. By asking the participants to gesture about specific features of the event, they could also shift the language that the speakers used. Also, the more information that there is to be packaged up into speech in a task, the more gestures people tended to use, to help them package up all that information.

Gesture explores cognition: Trying out ideas before you even say them

There have been a number of studies that show that when people are learning about a new concept (particularly school-age children), there is a phase where there is a ‘mismatch’ between what they’re talking about and their gesture. For example when a young child sees water from a tall, narrow glass poured into a wider glass, the child will often say there is less water in the second glass, because the level is lower. If the child makes a hand-spreading gesture while saying this though, it is an indication that the child is beginning to understand that there’s actually the same volume of water, held in a different shape. Difficult tasks can lead to a speech/gesture mismatch, which the authors argue is the brain trying out ideas that aren’t expressed in speech. They also suggest that half-finished gestures may be an indication that the brain is abandoning ideas,  indicating unsuccessful exploration.

My favourite quote from this part of the paper:

“The key features of these phenomena are that the microgenesis of ideas in gesture is, to some extent, independent from (and blazing the trail for) the microgenesis of ideas in speech, and that ideas develop in gesture via a process of trial and error” (p. 9)

Gesture is linked to other ‘practical actions’

The authors argue that gestures share features in common with other practical actions that we do with our hands and other parts of the body. They make a number of observations on this point:

People gesture more when talking/thinking about things that involve movement

Gesture rates also affected by how easy it would be to manipulate the objects being gestured about. In their example a picture of a spiky mug propted fewer gestures in a rotation task than a smooth mug (because who the heck would want to touch an ouchy mug?)

If people have experience manipulating the object being discussed then they gesture about it more

But, they note that there are some extra things that gestures can do for cognition that actions don’t. Firstly, because gestures have this ‘schematising’ function, they have a stronger influence on thought and they’re better for making generalisation while making relevant information more flexible and efficient. Secondly the authors argue that gestures can affect through more strongly than actions. In a map recollection task people who gestured to remember the route did better than people who drew it. They also discussed research on expert users of abacuses, who calculate faster with an imaginary abacus BECAUSE REAL ONES SLOW THEM DOWN TOO MUCH . Kita, Alibali and Chu didn’t write that all in capitals, but I’m still blown away by the idea that reality is holding back calculation.

They do note that it’s possible that sometimes schematic representations in gesture may not have the relevant information, or be distracting, but setting up an experiment to confirm this would be difficult.

But Lauren, don’t you always say that gesture and speech are closely linked???

I do, and they are. Kita, Alibali and Chu discuss this too:

“We argue that this occurs because the action generation system and the speech production system are highly interactive. As proposed by Kita and Özyürek (2003), the two systems can exchange information and align their contents. Thus, the contents of concurrent gesture and speech tend to converge.” (p. 11)

That is, gesture and speech are very closely related, but before that gesture and thought are very very closely related, and because speech is also linked to thought, it all tends to come together so quickly that it’s taken them decades of experiments to start pulling these relationships apart.

They also discuss how their model fits with some other research that looks at the relationship between gesture and language. For example, it’s been observed that gesture helps with retrieving words when you can’t quite remember them. The authors of this paper argue that this fits with their model, because the ‘activation’ function of gesture can retrieve a representation of an item or a feature of it, which then activates associated information such as the name for it.

There has been a great deal of research that indicates that gestures are useful for the people that we are speaking to, not just our own thinking. The authors of this paper don’t deny that that may be the case, instead arguing that it is possible for gestures to be both self-oriented and have a communicative function. Indeed, the schematisation that is beneficial for the speaker’s cognition may also be what is so useful about them for an interlocutor.

They also argue that this still works for objects that we have no direct experience interacting with, because we are also able to navigate a  “virtual environment” of experiences.

One final great quote from the paper that sums it all up

“By schematizing spatio-motoric information for these four functions, activation, manipulation, packaging and exploration, gesture facilitates cognitive processing and generates novel ideas, strategies and solutions that are easy to process, adaptable, and generalizable.” (p. 14)

Reference:

Kita, S., Alibali, M. W., & Chu, M. (2017, February 27). How Do Gestures Influence Thinking and Speaking? The Gesture-for-Conceptualization Hypothesis. Psychological Review. Advance online publication. http://dx.doi.org/10.1037/rev0000059