Ability to recognize familiar faces depends on a neural code shared across brains

The ability to recognize familiar faces is fundamental to social interaction. This process provides visual information and activates social and personal knowledge about a person who is familiar. But how the brain processes this information across participants has long been a question. Distinct information about familiar faces is encoded in a neural code that is shared across brains, according to a new Dartmouth study published in the Proceedings of the National Academy of Sciences.

"Within visual processing areas, we found that information about personally familiar and visually familiar faces is shared across the brains of people who have the same friends and acquaintances," says first author Matteo Visconti di Oleggio Castello, Guarini '18, who conducted this research as a graduate student in psychological and brain sciences at Dartmouth and is now a neuroscience post-doctoral scholar at the University of California, Berkeley. "The surprising part of our findings was that the shared information about personally familiar faces also extends to areas that are non-visual and important for social processing, suggesting that there is shared social information across brains."

For the study, the research team applied a method called hyperalignment, which creates a common representational space for understanding how brain activity is similar between participants. The team used data obtained from three fMRI tasks with 14 graduate students who had known each other for at least two years. In two of the tasks, participants were presented with images of four other personally familiar graduate students and four other visually familiar persons, who were previously unknown. In the third task, participants watched parts of The Grand Budapest Hotel. The movie data, which is publicly available, was used to apply hyperalignment and align participants' brain responses into a common representational space. This allowed the researchers to use machine learning classifiers to predict what stimuli a participant was looking at based on the brain activity of the other participants.

The results showed that the identity of visually familiar and personally familiar faces was decoded with accuracy across the brain in areas that are mostly involved in visual processing of faces. Outside of the visual areas however, there was not a lot of decoding. For visually familiar identities, participants only knew what the stimuli looked like; they did not know who these people were or have any other information about them.

In decoding personally familiar identities, the findings demonstrated that there was much more shared information across the brains of the participants. There was high decoding accuracy in four other areas outside of the visual system: the dorsal medial prefrontal cortex, which is known to be involved in social processing (processing other people's intentions and traits); the precuneus, an area which has been shown to be more active when processing personally familiar faces; the insula, which is known to be involved in emotional processing; and the temporal parietal junction, which plays an important role in social cognition and in representing the mental states of others (also known as the "theory of the mind").

"This shared conceptual space for the personal knowledge of others allows us to communicate with people that we know in common," says senior author Maria (Ida) Gobbini, a research associate professor in the Cognitive Science Program at Dartmouth and associate professor in the department of experimental, diagnostic and specialty medicine at the University of Bologna.

Past research by the team using fMRI experiments found that these "theory of mind" areas in the brain are activated when a person sees someone personally familiar. "When we see someone we know, we activate immediately who that person is," says Gobbini. "This is what allows us to interact in the most appropriate way with someone who is familiar." For example, how you interact with a friend or family member may be quite different from the way you interact with a colleague or boss.

It would have been quite possible that everybody has their own private code for what people are like but this is not the case. Our research shows that processing familiar faces really has to do with general knowledge about people."

James Haxby, co-author, professor of psychological and brain sciences, Dartmouth