VR for Cognition and Memory

This blog post is based on a recent book chapter “VR for Cognition and Memory” in Current Topics in Behavioral Neuroscience: Virtual Reality in Behavioral Neuroscience: New Insights and Methods. This work presents a review of research on VR’s ability to provide ecologically valid environments to study memory and cognition and discusses how features like interactivity, locomotion, and contextual control engage the brain’s memory systems more naturally than lab studies.

Revolutionizing Cognition Research with Virtual Reality

For decades, scientists have worked tirelessly to elucidate the intricate neural machinery supporting human cognition. This endeavor is certainly not for the faint of heart, as formidable challenges present themselves at every turn.

“To study cognition holistically means investigating interconnections between its rich repertoire of functions, including attention, reasoning, language, and memory. Memory is a particularly crucial facet, as it supports and subserves all other aspects of cognition; no cognitive task can be accomplished without memory.”

A holistic understanding demands that we study cognition as it operates in its natural habitat – the real world. Otherwise, as the parable of the blind men and the elephant warns, we risk gross mischaracterizations. Researchers must therefore conduct experiments in “verisimilar contexts (i.e. contexts appearing as the RW)” to achieve ecological validity.

Virtual reality (VR) presents an unprecedented opportunity in this regard. By simulating the real world, we can now study memory and cognition with enhanced veridicality.

“The environmental customization afforded by VR makes it an ideal tool for studying cognition in an ecologically valid fashion. Through the lens of memory studies, this chapter showcases the ways in which VR has advanced a meaningful and applicable understanding of cognition.”

The article presents a thorough review of research that showcases how VR is revolutionizing the study of cognition and memory.

Bridging the Gap Between Lab and Real-World Cognition

Traditional lab experiments often possess limited generalizability, whereas VR can provide naturalistic environments and tasks that echo real-world demands, easily bolstering ecological validity. Previous work has made a compelling case for how VR enhances the ecological validity of fMRI memory research.

VR experiences engage recollection-based memory retrieval akin to real events, unlike lab stimuli which rely more on familiarity. Indeed, VR experiences appear to be retrieved via recollection-based processes similar to those that support autobiographical/recollection memory, whereas retrieval of conventional screen experiences seems more similar to familiarity. This makes VR apt for integrated cognition and memory research.

VR Permits for Information to be Situation in Space

Most importantly, VR permits realistic navigation around virtual environments (c.f.), affording users with a sense of space (the scaffolding of memory). Both philosophers and psychologists alike postulate that brains have evolved solely to support purposeful and predictable movement. Many posit that the ontogeny of episodic memory relates to the onset of locomotion during infancy that scales with Hippocampal development (which also provides a mechanism for infantile amnesia and age-related episodic memory loss). One source of evidence to support this proposition is in the life cycle of the bluebell tunicate. This filter feeder begins to digest a substantial chunk of its cerebral ganglion once identifying a suitable undersea perch to spend the rest of its existence. This phenomenon suggests that once it has served its purpose as a neural network supporting movement, the cerebral ganglion yields greater utility to the organism as nutrition.

From chemotaxis to cognitive maps, a representation of space is necessary for meaningful movement. A neural instantiation of a map that provides spatial bookmarks of an organism’s experiences, demarcating the locations of nutrition and enemies within an environment, is a fundamental component of brains. Indeed, there is a primacy of spatial content in the neural representation of events. Spatial information is often recalled earliest in the retrieval process, and the degree to which individuals report confidence in their autobiographical memories is predicted by their knowledge of the spatial layout of the setting in which the memory occurred. The Method of Loci (a.k.a. Memory Palace) mnemonic has long been appreciated for its ability to increase memory by imagining to-be-remembered information placed at familiar locations. Past work used a VR implantation of this technique to suggest that the principal component behind mnemonic efficacy is the explicit binding of the objects to a spatial location and revealed a tight relationship between spatial memory (SM) and free recall of encoded objects. These observations showcase that space and memory are inextricably linked at conceptual and neuronal levels – a notion that has become entrenched in popular culture; the phrase “out of space” is often used when indicating a computer’s memory is full.

If space is the inescapable wallpaper that serves as the backdrop for all experience, then it follows that as our spatial or environmental context changes, so should the neural activity underlying diverse cognitive processes. Given that VR can easily change environments, it provides an unparalleled landscape with which to study the intersection of space, memory, and cognition.

Additionally, VR enables human analogs of spatial memory research previously limited to animal models, like virtual radial arm mazes. This facilitates powerful translational research from rodents to humans.

Key Features of VR That Facilitate Cognition Research

Below are some features highlighted by the chapter that are exclusive to VR. Such features permit real-world scenarios with increased experimental control and significantly less costs.

• VR provides absolute control over the environment. This permits isolation and systematic manipulation of spatial contexts, immersion, emotions, embodiment, etc.
• Rapid teleportation between environments induces robust context-dependent learning, a fundamental principle in memory encoding.
• Interactivity and locomotion increase embodiment and navigational involvement, enhancing hippocampal memory systems.
• Implicit metrics like gaze, paths, and object interactions generate objective measures of memory and attention unbiased by subjective reporting.
• Brain imaging during VR reveals in vivo neural correlates of cognition impossible with real-world navigation.
• VR spatial mnemonics such as the Method of Loci can provide performance improvements over just imagination by standardizing and controlling the environments.

Applications of VR for Assessing and Enhancing Cognition

Conventional measures of memory typically focus on core content (i.e., the “what”) instead of the true binding that happens in actual episodes (i.e., “what,” where,” and “when”). They also often use verbal materials, which makes the test sensitive to performance in non-memory domains, permitting for compensatory strategies which could erroneously reveal normal “memory.” Subjective reports rarely scale with performance on traditional memory tests, warranting criticism that such measures wrongly estimate memory capacities for everyday situations. For example, patients reporting topographical memory deficits have preserved ability in tabletop tests of spatial or geographical knowledge. Additionally, cognitive complaints in amnesiacs typically show little correlation with verbal memory tests used in clinical settings.

VR tasks, however, have been more reliable in tracking self and caregiver reports of deficits that impact quality of life. The points below highlight other aspects of VR that can increase the ecological validity of both the detection and amelioration of memory deficits.

• VR scenarios like virtual stores and routes enable sensitive, ecologically valid tools to identify mild cognitive impairment early.
• VR spatial navigation paradigms can differentiate Alzheimer’s from milder impairment based on hippocampal recruitment patterns.
• VR enables safe exposure therapy for memory deficits induced by trauma and realistic training for brain injury rehabilitation.
• Spatial mnemonic techniques adapted to VR boost memory beyond baseline abilities in healthy individuals.
• VR puzzles engage aging minds, increasing motivation. Long-term regimes may prevent decline. As one study found, “6 months of VR training powerfully increased long-term recall.”
• VR training could augment real-world cognition and rehabilitate deficits, with proven memory transfer effects.

In conclusion, VR enables an unprecedented ability to understand real-world cognition, precisely diagnose impairments, and develop interventions that enhance memory and cognition. The immersive, interactive nature of VR environments engages our brains’ memory systems far more naturally than traditional lab studies.

The inherently engaging qualities of VR, coupled with its ability to implicitly quantify and enhance memory, make it a powerful tool in populations spanning from pediatrics to the elderly.

Indeed, VR may catalyze discoveries about the very mechanisms underlying human consciousness itself, which intimately relies on episodic memory. By augmenting these processes, VR could profoundly transform our experience and understanding of consciousness. The future of cognition research has never looked more exciting.