Wang & Sun (2025)

Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Length

arXiv

Find It

DOI

Abstract

Information retrieval in Large Language Models (LLMs) is increasingly recognized as intertwined with generation capabilities rather than mere lookup. While longer contexts are often assumed to improve retrieval, the effects of intra-context interference remain understudied. To address this, we adapt the proactive interference (PI) paradigm from cognitive science, where earlier information disrupts recall of newer updates. In humans, susceptibility to such interference is inversely linked to working memory capacity. We introduce PI-LLM, an evaluation that sequentially streams semantically related key-value updates and queries only the final values. Although these final values are clearly positioned just before the query, LLM retrieval accuracy declines log-linearly toward zero as interference accumulates; errors arise from retrieving previously overwritten values. Attempts to mitigate interference via prompt engineering (e.g., instructing models to ignore earlier input) yield limited success. These findings reveal a fundamental constraint on LLMs' ability to disentangle interference and flexibly manipulate information, suggesting a working memory bottleneck beyond mere context access. This calls for approaches that strengthen models' ability to suppress irrelevant content during retrieval.

Reference

Wang, C., & Sun, J. V. (2025). Unable to forget: Proactive interference reveals working memory limits in LLMs beyond context length (arXiv:2506.08184v3). arXiv. https://doi.org/10.48550/arXiv.2506.08184

What Rests on This

4 mechanisms cite this source. Each finding below is quoted as it appears on that mechanism's own page.

Working Memory Memory Systems

Measured in AI

Adapting the human proactive-interference paradigm to 30+ models, a workshop paper found retrieval of the latest value of a repeatedly updated item declined log-linearly toward zero as earlier values accumulated; larger models resisted longer, while context-window length had no significant effect (no human data)

Reconsolidation Memory Processes

Measured in AI

A workshop paper adapted the human A-B, A-C proactive interference design to the conversation: as a value was repeatedly updated, models' accuracy at reporting the latest value declined log-linearly toward zero because they retrieved overwritten values; larger models held up better, but no human participants were tested

Retrieval Interference Memory Processes

Measured in AI

Proactive interference tests in AI have no human group: in a 2025 preprint (ICML workshop), dozens of language models tracking 46 repeatedly updated keys lost accuracy log-linearly toward zero as overwritten values piled up, even with the answer just before the question; the paper states that people do well on such tasks but tested none

Motivated Forgetting Memory Processes

Measured in AI

No study located gives a language model the directed forgetting or think/no-think procedure, or tests people and AI on the same forget-on-instruction task; AI work instead appears under other names (in-context unlearning, machine unlearning, memory deletion), so the AI level is indirect
Measured in AI

Current conversation: in a 2025 workshop paper, 30+ language models tracked repeatedly updated values in their context window (what the model can see in the conversation); explicitly telling models to forget earlier values improved accuracy by under 10 percentage points, and all models declined as outdated values piled up, with larger models declining more gradually