Study explores impact of active navigation and immersive environments on episodic memory

Just like a time machine, episodic memory enables us to travel to the past in our minds and relive personal experiences with full clarity, as if we were going through them all over again. These can range from remembering where we left the car keys last night to the last time we saw a relative that we haven't seen in years. This ability to record daily experiences starts to deteriorate early in people with neurodegenerative diseases such as Alzheimer's disease.

A team of researchers at the UOC (Universitat Oberta de Catalunya) has now carried out a study on episodic memory performance in a specific environment and its relationship with locomotion, body movement and environmental characteristics in an environment equipped with new immersive technologies, in order to establish how memories of lived experiences are created.

"The study focuses on how episodic memory works, more specifically on how the spatial environment affects how and how effectively memories are organized in our minds," said Álvaro Pastor, an architect, cognitive scientist, researcher at the UOC's XR-Lab and Learning, Media and Entertainment (GAME) research group, and one of the lead authors of the study together with Pierre Bourdin-Kreitz, a member of the teaching and research staff at the UOC's Faculty of Computer Science, Multimedia and Telecommunications and coordinator of the XR-Lab at the same university.

The study, which has been published in open access in the Scientific Reports journal (Nature group), seeks to establish whether active navigation (when a person moves around a given environment) has a bearing on how episodic memory works and to study how the physical characteristics of their spatial environment can modulate the way episodic memory is organized.

The authors carried out a series of tests using immersive technologies at Barcelona's CaixaForum museum. An analysis of the data suggests that such technologies have potential applications in the design of non-invasive treatments and therapies for neurodegenerative diseases.

The researchers point out that the relationship between space and episodic memory has been the subject of scientific research for decades. In fact, the method of loci or mind palace, one of the oldest mnemonic rules and among the most reliably proven to work, is based on creating a mental building and associating pieces of information with specific places within it.

Creation of a cognitive map 

During the study, the 28 participants moved around the museum's two floors, which were connected by a set of stairs, either passively using virtual reality or actively using augmented reality. During their tour, they were shown a number of images in specific locations on each floor.

Afterwards, the authors tested how much each participant could remember both immediately after the tour and 48 hours later. According to the authors, "the results suggest that those participants who covered the route on foot remembered more information in both tests."

Furthermore, when looking at the location of the most memorable images, the authors found that participants recalled more information from the place where they had changed floors: the stairs. "Information learned when next to the museum stairs was more memorable than that encountered in the middle or at the end of the route, even if it was close to colourful artworks," said the authors.

After analysing the data and the subjects' responses, the authors noted that active learning experiences may lead to better episodic memory performance than passive encoding.

Actively navigating through an environment enables us to collect enough information for our episodic systems to build a sort of cognitive map of our experience, leading to more effective subsequent recall."

Álvaro Pastor, architect, cognitive scientist, researcher at the UOC's XR-Lab and Learning, Media and Entertainment (GAME) research group

Furthermore, the physical elements involved in active navigation, whether they help or hinder it, appear to be more strongly integrated in our memory of the experience. In fact, the data show that information and experiences acquired in physical parts of the museum, such as the stairs, are remembered better than stimuli from other places. "This peculiarity suggests that, when recording the experience, our cognitive map places a particular focus on the most salient aspects of spatial navigation and creates associations between them and the information learned when in physical proximity to them," said the UOC researcher.

Use of artificial intelligence

The study also looked at the type of image displayed in the tour of the museum that participants had been told to remember. These images were frontal portraits of artificial human faces specially generated for this study using neural network-based AI.

"By creating artificial faces, we were able to provide participants with images they'd never seen before and ensure their features were uniform, including their facial expression and lighting," said Pastor, who also noted that the high-sensitivity cognitive tasks involved in episodic memory include remembering a person's face after meeting them for the first time and the contextual details of the encounter, as well as its time and place.

Memory enhancement applications 

As for the practical applications of the study, the research has led to the development of a novel and sound method involving the use of immersive technologies to better assess episodic memory under natural conditions akin to real-world conditions. This means that scientifically proven information is available to help design therapies involving active exploration through enriched spatial environments.

“In the case of healthy individuals, using the results of this study in active learning experiences with immersive apps can help maximize the memorability of information at specific locations of a course," said Bourdin-Kreitz.

However, the benefits of active learning do not stop at educational design. "Taking part in active learning experiences that involve a person's whole body instead of learning passively could provide a preventive measure to help healthy people retain their episodic memory abilities for longer," said Pastor.

"Just as with the method of loci or mind palace, the idea of walking while learning to increase retention may have been known to mankind since ancient times, at least if we are to believe the story that Aristotle taught his students philosophy while walking through the streets of Athens, and thanks to immersive technologies we can now study this phenomenon in depth,” said Bourdin-Kreitz.

As for clinical applications, virtual and augmented reality technologies can be used to help design rehabilitation programmes tailored to each patient. "Clinical interventions based on such technologies may be able to slow down the progression of the disease in an inexpensive and easily scalable way and, as they are non-invasive, they may help increase patient adherence to treatment without jeopardizing safety," concluded the authors after noting that they are continuing to work on new research to, for example, add smells in a controlled way while simultaneously showing subjects virtual images on a virtual reality headset.