How do our memories take shape?

(Press-News.org) Your brain is constantly evaluating which aspects of your experiences to either remember for later, ignore, or forget. Dartmouth researchers have developed a new approach for studying these aspects of memory, by creating a computer program that turns sequences of events from a video into unique geometric shapes. These shapes can then be compared to the shapes of how people recounted the events. The study provides new insight into how experiences are committed to memory and recounted to others. The results are published in Nature Human Behavior and were based on how people remembered the experience of watching an episode of Sherlock, a BBC television show.

"When we represent experiences and memories as shapes, we can use the tools provided by the field of geometry to explore how we remember our experiences, and to test theories of how we think, learn, remember, and communicate," explained senior author Jeremy R. Manning, an assistant professor of psychological and brain sciences, and director of the Contextual Dynamics Lab at Dartmouth. "When you experience something, its shape is like a fingerprint that reflects its unique meaning, and how you remember or conceptualize that experience can be turned into another shape. We can think of our memories like distorted versions of our original experiences. Through our research, we wanted to find out when and where those distortions happen (i.e. what do people get right and what do people get wrong), and examine how accurate our memories of experiences are," he added.

The Dartmouth research team examined a public dataset containing brain recordings from 17 people who had watched the Sherlock episode and then described what had happened in their own words. The dataset also contained detailed scene-by-scene annotations of the episode. The team ran those annotations through their computer program to identify 32 unique topics or themes that were present in each moment of the episode. Through computer modeling, the researchers then created a "topic model" of the episode, which was comprised of 32 dimensions to reflect each thematic topic. Different moments of the episode that reflected similar themes were assigned to nearby locations in the 32-dimensional space. When these results are visualized in 2D, a connect-the-dots-like representation of successive events emerges. The shape of that representation reflects how the thematic content of the episode changes over time, and how different moments are related. The researchers used an analogous process to obtain the shapes of how each of the 17 participants recounted the events of the episode.

When the geometric shapes representing the Sherlock episode were compared to the shapes representing a participant's recounting of it, the researchers were able to identify which aspects of the episode people tended to remember accurately, forget or distort. The coarse spatial structure of the episode's shape reflects the major plot points and acts like a building's scaffolding. The shape of every participant's recounting reproduced this coarse-scale scaffolding, indicating that every participant accurately remembered the major plot points. The episode's shape also comprises finer-scale structure, analogous to architectural embellishments and decorations, that reflected specific low-level conceptual details. Some participants accurately recounted many of those low-level details, whereas others recounted only the high-level plot points.

"One of our most intriguing findings was that, as people were watching the episode, we could use their brain activity patterns to predict the distorted shapes that their memories would take on when they recounted it later," explained Manning. "This suggests that some of the details about our ongoing experiences get distorted in our brains from the moment they are stored as new memories. Even when two people experience the same physical event, their subjective experiences of that event start to diverge from the moment their brains start to make sense of what happened and distill that event into memories."

The research team plans to apply their approach to other domains, including in health and education, as their methods of modeling the shapes of memories could be used to provide a more nuanced way of assessing if a patient will understand or remember what their doctor is telling them, or whether a student understands specific concepts in a course lecture.

INFORMATION:

Manning is available for comment at: jeremy.r.manning@dartmouth.edu. Andrew C. Heusser, a former postdoctoral researcher in the Contextual Dynamics Lab, who is now working for Akili Interactive and Paxton C. Fitzpatrick '19, lab manager and research assistant in the Contextual Dynamics Lab at Dartmouth, served as co-first authors of the study.