New insights into the way the brain blends and balances information from different senses

Jun 12 2004

New research has found that the brain continues to accept ambiguous visual information about an object in motion even when it conflicts with more reliable information that we can touch. The studies, which appear in the June 7 issue of the journal Psychological Science and the forthcoming June issue of the journal Cognitive, Affective, & Behavioral Neuroscience, provide new insights into the way the brain blends and balances information from different senses.

The research, conducted by Vanderbilt psychologists Randolph Blake, Centennial Professor of Psychology, and Thomas W. James and Kenith V. Sobel, research associates, found that the region of the brain that specializes in processing visual movement-the middle temporal visual center, or MT-also responds to motion that we feel. But they were surprised to discover that when individuals were presented with an ambiguous visual image and were able to touch that object, their brains did not fuse the visual and touch inputs into a single, accurate representation. Instead, the researchers found that the brain keeps the two inputs separate and accepts a degree of "cognitive dissonance" when the two conflict.

"This suggests that there is naturally a higher level of inconsistency between seeing and feeling something moving than there is between seeing and feeling something's shape," says James.

To explore the relationship between vision and touch, the team developed a procedure that relies on a well-known visual illusion called the kinetic depth effect. The kinetic depth effect begins with a group of dots arranged in a circular pattern on a computer screen. The dots are programmed so that half move left to right and half move right to left. The dots' movements are choreographed so that they move as if they are fixed to the surface of a transparent globe. This tricks the brain into seeing them as forming a rotating, three-dimensional sphere. The viewer cannot tell from the dots in which direction the sphere is turning. So the brain splits the difference-the sphere appears to rotate from right-to-left 50 percent of the time and from left-to-right 50 percent of the time.

The researchers set up a rotating Styrofoam ball that the subjects could touch with both hands and projected the moving dot illusion into their eyes in such a way that it appeared to be the same in every way as the physical ball.

"Our thought was that if a person touched something turning in a specific direction, then the perceptual system should fuse the tactile and visual information, resulting in people 'seeing' a sphere rotating in a direction that was consistent with the more reliable source of sensory information," says James.

They found instead that the tactile and visual information did not fuse. The tactile information did, however, change the subjects' perception of which direction the sphere was rotating. The amount of time that the subjects reported the visual sphere rotating in the same direction as the physical sphere jumped from 50 percent to 65 percent. Yet it still appeared to rotate in the opposite direction 35 percent of the time, in direct contrast to what they were feeling with their hands.

The researchers went a step further and monitored brain activity in the part of the brain most likely to be involved, the Middle Temporal visual center, using functional magnetic resonance imaging. This technique detects activity levels in different parts of the brain by measuring blood flow. They recorded a reliable increase in activity in MT when the globe was rotating compared to when it was stationary for both vision and touch. But the activity stimulated by the visual stimulus was four times greater than that generated by touch.

A central question remained unanswered: Why didn't the visual and tactile information fuse? The researchers speculated that the fact that subjects could not see their fingers touching the rotating sphere might have been enough to keep the fusion from taking place.

So they came up with another approach, which is described in the Cognitive, Affective, & Behavioral Neuroscience article. They attached a wire-frame sphere to a motor to enable it to slowly rotate. When viewed with one eye closed, it created the same illusion as the moving dots. In this case, however, subjects were able to reach out and directly touch the sphere as they were looking at it to determine in which direction it was rotating.

The researchers were surprised to find that this procedure produced almost precisely the same results as their first effort. Without touching the sphere, subjects reported that it appeared to rotate in each direction approximately 50 percent of the time. When the subjects were touching the sphere, it appeared to them to rotate in the direction consistent with what they were feeling 60 to 70 percent of the time, while appearing to rotate in the opposite direction 30 to 40 percent of the time.

"It is surprising that an unreliable visual stimulus should be that resistant to tactile input," says James. "This points out that, in terms of visual motion processing, the mechanism that produces alterations in the perceptual state must be very powerful because it is not over-ridden even by highly reliable tactile input." For more news about Vanderbilt research, visit Exploration, Vanderbilt's online research magazine, at http://exploration.vanderbilt.edu.