Mnemonic State Decoding

The mnemonic state task is a variation on the traditional AB/AC (or AB/AD) paradigm. Participants view two lists of images of categorically-related common objects. For example, if a participant sees a bench in the first list, they will see a different bench in the second list. In essence the first bench constitutes the “B” item, the second bench constitutes the “C” item, and their shared category (bench) constitutes the “A” cue. The critical manipulation is that during the second list, we tell participants whether they should encode the currently presented bench or retrieve the previously presented bench. In this way, we explicitly bias participants to encode or retrieve.

Encode/retrieve trials are as closely matched as possible:

• Perceptual input is the same: every trial includes a “new” (not previously presented) object image that is categorically related to an object image previously presented in List 1
• Motor output is the same: participants do not make any responses during either encode or retrieve trials
• Judgments/decisions are the same: participants do not make any decisions about the object images

The short answer is: we don’t. However, the retrieval state (or mode, Tulving 1983) is thought to be a necessary precursor of, but distinct from, successful retrieval. In other words, you engage the retrieval state in the attempt to retrieve regardless of whether you are ultimately successful in that attempt. That we find retrieval state modulations in the absence of episodic memory demands (Long 2023) is consistent with this framing of the retrieval state.

We test memory for all items at the end of the session. Better memory for List 2 items paired with the “NEW”/encode instruction relative to List 2 items paired with the “OLD”/retrieve instruction indicates that participants differentially processed those items in response to the instruction cues.

Although we have collected data from over 100 participants in the mnemonic state task, not all participants show robust mnemonic state decoding. There are multiple potential reasons for why within-participant classification may fail — some more interesting than others. Poor data quality, either in terms of participant behavior or in terms of noisy brain signals, will negatively impact classifier performance. We typically exclude participants whose data are of poor quality, so this is less likely to account for low classifier performance. Instead, we think that some participants might not engage mnemonic brain states in exactly the same way (i.e. using the exact same substrates) across every trial and even throughout the trial. Thus, our classifier, which is trained on all trials over the full stimulus interval (2000 ms) may not be able to distinguish encoding and retrieval in such participants. However, this does not preclude the possibility that shared substrates underlie mnemonic states; rather, variable recruitment of those substrates may relate to important individual differences.

The premise that encoding and retrieval tradeoff stems from rodent and theoretical work conducted by Michael Hasselmo and colleagues showing that differential — and specifically opposing — hippocampal circuitry support encoding and retrieval. Thus, at the level of the hippocampus, the two states cannot be engaged in simultaneously. The ability to switch between these two states is a critical open question that may account for a host of behavioral effects across cognition and the lifespan — we are currently pursuing projects related to this question.

The critical distinction here is that of representations vs. processes. We expect the processes of encoding and retrieval to differ, but not necessarily the representations. Given a long history of evidence to suggest that the regions which support the perception of a stimulus also support the knowledge for and memory of that stimulus (e.g. Allport 1995), the representations of a stimulus may be preserved between encoding and retrieval. Such an account makes sense from a ‘storage’ perspective of the brain – why would we have two completely distinct systems to represent an external percept (encoding) and an internal image (retrieval)? However, it is important to note that there is recent evidence to suggest that representations may be transformed between the two states (e.g. Favila et al. 2020; Long & Kuhl 2021; Steel et al. 2024).

We believe that assigning specific features to mnemonic states is antithetical to the conceptual idea of a ‘brain state.’ First there is the practical consideration that as we use regularized logistic regression, the classifier is designed to create sparse weights. As a consequence of this design, if there are two highly correlated features which respond similarly for one condition (e.g. retrieval), only one of the two features should end up being leveraged by the classifier. Thus, feature weights do not tell us how the brain solves a problem/carries out a function, but instead tell us what information the classifier is using, which is not something in which we are especially interested. Second, although univariate approaches have been immensely valuable in laying the foundation for much of our current understanding of how the brain gives rise to the mind, we seek to advance understanding by moving beyond these approaches to investigate whole-brain and network-level effects. Although technically possible, a direct contrast of signals between encode and retrieve trials would leave us with hundreds to thousands of univariate comparisons to reconcile and interpret, significantly detracting from the utility of multivariate methods. Ultimately, we do not expect brain states to be supported by a single region and/or frequency.