Jointly Considering Past and Future Locations in Visual Working Memory

Core Concepts

Both past and future memory attributes are retained and can be accessed together when memory contents become relevant for behavior.

Abstract

The study investigated how the brain processes visual working memory contents when the encoding (past) and to-be-tested (future) locations are dissociated. The key findings are: Gaze shifts showed biases towards both the past (encoded) location and the future (to-be-tested) location of the cued memory item, with these biases emerging at overlapping time windows. Analysis of individual saccades revealed that saccades were jointly biased towards both the past and future locations, suggesting a truly joint consideration of past and future memory attributes at the single-trial level. This suggests that the brain simultaneously retains copies of both the past and future-relevant locations in working memory, and reactivates each during mnemonic selection, rather than serially considering past and then future. The findings reveal the rich nature of working memory representations, where both past and future memory attributes are preserved and can be accessed together when memory contents become relevant for behavior.

Stats

"Working memory enables us to hold onto past sensory information in service of upcoming future behaviour." "Participants were able to perform this dynamic visual working-memory task, with an average accuracy of 70±2 percent correct (mean ± SEM) and an average reaction time of 1218 ± 125 ms."

Quotes

"Accordingly, by its very nature, working memory is concerned with two components: the past and the future." "Strikingly, this future bias also emerged early after the cue. An overlay of the spatial biases (toward vs. away) in the orthogonally manipulated past and future axes (Fig. 1d), revealed consideration of past and future locations at overlapping time windows." "Critically, this visualisation and quantification enabled us to disentangle the two alterative sketched above in which past and future saccades at the single-trial (individual-saccade) level were either independent (past bias regardless of future and future bias regardless of past) or dependent (joint past and future bias)."

Key Insights Distilled From

Jointly looking to the past and the future in visual working memory

by Liu,B., Alex... at www.biorxiv.org 02-02-2023

https://www.biorxiv.org/content/10.1101/2023.01.30.526235v3

Deeper Inquiries

How might the joint retention of past and future memory attributes contribute to the flexibility and robustness of working memory representations in dynamic, real-world situations

The joint retention of past and future memory attributes in working memory can significantly contribute to the flexibility and robustness of representations in dynamic, real-world situations. By simultaneously maintaining information about where an object was last seen (past location) and where it is expected to reappear (future location), individuals can effectively navigate changing environments and respond to shifting demands. This dual representation allows for seamless transitions between past experiences and future actions, enabling quick and adaptive decision-making. In real-world scenarios, such as the example of capturing a photograph of a bird that disappeared behind a building, the joint consideration of past and future memory attributes becomes crucial. By retaining both the past location where the bird was last seen and the future location where it is expected to reappear, individuals can efficiently plan their actions, anticipate the bird's movement, and adjust their gaze or attention accordingly. This flexibility in working memory representations allows individuals to maintain a continuous flow of information processing, integrating past experiences with future goals. Moreover, the joint retention of past and future memory attributes enhances the robustness of working memory representations by providing redundancy and backup information. If one aspect of the memory (e.g., past location) becomes less relevant or accurate over time, the complementary aspect (e.g., future location) can compensate and ensure the continuity of cognitive processes. This redundancy minimizes the risk of information loss or degradation, especially in dynamic and unpredictable environments where rapid adjustments are necessary. Overall, the joint consideration of past and future memory attributes in working memory enables individuals to adapt to changing circumstances, anticipate future events, and maintain a coherent representation of the environment, enhancing cognitive flexibility and robustness in real-world situations.

What are the potential costs or tradeoffs associated with simultaneously maintaining multiple spatial frames of reference (past and future) for working memory contents, compared to a more parsimonious single-frame representation

Simultaneously maintaining multiple spatial frames of reference, such as past and future locations, for working memory contents can introduce potential costs or tradeoffs compared to a more parsimonious single-frame representation. While the joint consideration of past and future memory attributes offers advantages in terms of flexibility and robustness, it also poses challenges and cognitive demands that may impact cognitive processing efficiency and resource allocation. One potential cost of maintaining multiple spatial frames of reference is the increased cognitive load required to process and update information across different frames. Juggling between past and future locations may consume additional cognitive resources, leading to higher mental effort and potential cognitive fatigue, especially in complex or demanding tasks. This increased cognitive load could result in slower processing speed, reduced accuracy, or higher susceptibility to errors, particularly when the task involves rapid changes or conflicting information. Furthermore, the simultaneous consideration of multiple spatial frames may introduce interference or competition between past and future representations. In situations where past and future locations conflict or overlap, individuals may experience interference effects that hinder accurate memory retrieval or decision-making. Resolving conflicts between past and future information could require additional cognitive control mechanisms, leading to delays or inefficiencies in processing. Moreover, maintaining multiple spatial frames of reference could potentially limit the capacity or precision of working memory. Dividing attention between past and future locations may reduce the resources available for encoding, storing, or manipulating other types of information, such as object identity or temporal order. This tradeoff between spatial and non-spatial features could impact the overall quality and coherence of working memory representations, affecting task performance and cognitive flexibility. In summary, while the joint consideration of past and future memory attributes enhances flexibility and robustness, it also introduces cognitive costs and tradeoffs related to increased cognitive load, interference effects, and potential limitations in capacity or precision. Balancing these factors is essential for optimizing cognitive processing efficiency and performance in dynamic, real-world situations.

Could the joint consideration of past and future memory attributes extend beyond spatial information to other feature dimensions, such as object identity or temporal order, and how might this impact our understanding of the structure and dynamics of working memory representations

The joint consideration of past and future memory attributes in working memory may extend beyond spatial information to other feature dimensions, such as object identity or temporal order, offering new insights into the structure and dynamics of working memory representations. By incorporating multiple dimensions of information simultaneously, working memory can create more comprehensive and contextually rich representations that support complex cognitive processes and adaptive behavior. In terms of object identity, the joint consideration of past and future attributes could involve retaining not only the spatial location of an object but also its specific features, such as color, shape, or texture. By integrating information about both where an object was encountered in the past and where it is expected to appear in the future, individuals can maintain a holistic representation of the object, facilitating recognition, categorization, and decision-making. This multi-dimensional approach to object representation enhances the discriminability and specificity of working memory contents, enabling more accurate and detailed mental representations. Similarly, the joint consideration of past and future memory attributes may apply to temporal order information, allowing individuals to track the sequence of events or actions over time. By encoding both the past occurrence and the anticipated future occurrence of events, working memory can support temporal organization, prediction, and planning. This temporal dimension adds a dynamic aspect to memory representations, enabling individuals to anticipate upcoming events, coordinate actions, and maintain temporal coherence in cognitive processing. Overall, extending the joint consideration of past and future memory attributes to other feature dimensions beyond spatial information enriches the complexity and flexibility of working memory representations. By integrating multiple dimensions of information in a cohesive manner, working memory can support a wide range of cognitive functions, from object recognition to temporal planning, and provide a more comprehensive understanding of how memory operates in dynamic, real-world contexts.