Mechanisms
Memory Processes · Degradation
Forgetting
Example
You don't remember what you had for lunch three Tuesdays ago. Not a failure: nothing has come back since to make that worth keeping, and the system bet accordingly.
Forgetting in Humans
Definition
Forgetting is the loss of access to stored content as a systematic function of how likely that content is to be needed. Human retention functions match the statistical structure of the environment, meaning the odds an item will be called for given how often and how recently it has appeared, so forgetting reflects an adaptive bet rather than simple decay (Anderson & Schooler, 1991). It is now generally treated as actively produced rather than passive: stored traces are weakened or eliminated by identifiable biological processes (Richards & Frankland, 2017), and their accessibility is reduced by control processes recruited when a memory interferes with what a person is currently doing (Anderson & Hulbert, 2021). The two accounts describe different levels of the same outcome rather than competing explanations of it.
Scope: This row covers ordinary, incidental forgetting, meaning loss of access that occurs without the person intending it. Deliberate forgetting, including directed forgetting and retrieval suppression, is a separate construct and is not treated here (Anderson & Hulbert, 2021). Retrieval-induced forgetting falls inside this row's scope, since the loss is a byproduct of ordinary retrieval rather than its aim, but whether it is caused by targeted inhibition is disputed (Potter et al., 2018).
Neural Substrate
Human evidence for active forgetting comes chiefly from control processes acting on memory. Anderson and Hulbert (2021) review a body of work in which the lateral prefrontal cortex, especially on the right, exerts top-down modulation over hippocampal and cortical activity, reducing the accessibility of memories whose continued availability disrupts a current goal. The most direct demonstration is Wimber et al. (2015), an fMRI study of 24 healthy adults who learned two pictures to each of a set of cue words and then repeatedly retrieved one of them. Using perceptual templates for each individual picture, the authors tracked the competing memory across four retrievals and found that its pattern in ventral visual cortex was progressively driven below the level of uncued baseline items, that mid-ventrolateral prefrontal engagement predicted the size of that suppression, and that the suppression predicted which items were later forgotten. On the final recognition test, competitors were recognised at 75.2% against 82.1% for matched baseline items (t23 = 4.91, p \< 0.001). This finding is contested. Potter et al. (2018) obtained the original data, showed that the interaction supporting a causal reading (p = .041) depended on scoring unanswered trials as errors in a two-alternative task where chance is 50%, and failed to reproduce any forgetting effect across replication attempts including a preregistered one with twice the original sample.
The cellular mechanisms of forgetting are characterised almost entirely in non-human animals. Richards and Frankland (2017) review evidence that potentiated synapses are weakened by endocytosis of GluA2-containing AMPA receptors, eliminated through Rac1-mediated spine shrinkage in flies and mice and through Arp2/3 destabilisation in C. elegans, and overwritten by hippocampal neurogenesis, with rodent studies showing bidirectional effects in which raising neurogenesis after training accelerates forgetting and suppressing it slows forgetting. Whether the neurogenic mechanism has a human counterpart is unsettled, and the two most-cited studies reach opposite conclusions from similar immunohistochemistry. Sorrells et al. (2018) examined 59 human hippocampal specimens and found no doublecortin-positive, PSA-NCAM-positive young neurons in any of 17 adult postmortem cases aged 18 to 77, concluding that a proliferative subgranular zone never forms in humans. Boldrini et al. (2018), using stereology on whole autopsy hippocampi from 28 individuals aged 14 to 79, found thousands of such cells per dentate region with no decline across the age span, alongside a smaller quiescent progenitor pool and reduced angiogenesis in older brains. The row therefore treats the control-based evidence as human and the decay and neurogenesis evidence as non-human, with the human status of the latter mechanism open.
AI at a Glance
Human forgetting withdraws access to stored content in proportion to how unlikely that content is to be needed again, tracking the frequency and recency structure of the environment (Anderson & Schooler, 1991), and it is produced by active mechanisms rather than passive decay (Richards & Frankland, 2017). No assessed AI mechanism estimates future need: context windows lose earlier material as a function of distance (Liu et al., 2024), memory layers delete on an elapsed-time schedule (Zhong et al., 2024), and test-time memory weights decay under capacity pressure and prediction error (Behrouz et al., 2024).
Forgetting in AI
How Far It Has Been Measured
Measurement in AI: Indirectly Measured
AI Memory Performance: Comparison Not Possible
Where It Lives in the System
What Has Been Measured
- In training, a model's recall of a fact tracks how often it appeared: across GPT-Neo, BLOOM and GPT-3 models, question-answering accuracy correlated strongly with the number of relevant documents in the training data, and larger models helped only slowly with rarely seen facts; this is frequency, one half of the need signal, measured under another name (Kandpal et al., 2023).
- Using monthly Wikipedia page views as a stand-in for how often an entity comes up, ten language models were much less accurate on facts about less popular entities, and scaling up mainly improved recall of popular ones (Mallen et al., 2023).
- Tracking facts inserted during pretraining of OLMo models (1B and 7B) showed retention falling linearly with the logarithm of later training steps, the same curve family that fits human retention, and facts repeated verbatim were forgotten faster than paraphrased ones; no human data were placed on the same scale (Chang et al., 2024).
- Image, speech and language models were found to forget the specifics of training examples seen early in training, so examples seen recently are the ones most at risk of being recalled; this is a recency effect in training, but loss here is driven by further training rather than by how likely the content is to be needed (Jagielski et al., 2023).
- In the conversation (the context window, meaning the text the model can see right now), models used information best at the start or end of a long input: with 20 documents GPT-3.5-Turbo scored 53.8% when the answer sat in the middle, below its 56.1% with no documents at all; the authors link this U-shape to the human serial-position effect but tested no people (Liu et al., 2024).
- A 2023 preprint probing GPT-J found only a slow decline in recall as neutral filler text was added between a fact and the question, a clear benefit when a repetition was spaced rather than massed, and much larger losses from interfering facts; comparisons with human data were qualitative (Janik, 2023).
- In saved memory, agent systems build in need-like forgetting: MemoryBank weakens or reinforces stored memories using an Ebbinghaus-style curve based on elapsed time and significance, and Generative Agents rank memories partly by a recency score that decays by a factor of 0.995 per game hour since last use; neither paper tests whether this forgetting tracks actual need or human retention, so this place is unmeasured (Zhong et al., 2024; Park et al., 2023).
- The one same-test comparison of retention in the conversation gave 100 Prolific participants and GPT-2 Small (2019) text spans repeated up to three times; people saw only the previous 10 words when predicting, while the model saw everything, and it reached near-perfect prediction after one presentation while people improved modestly (human-model correlation fell from r = .87 to .05); the gap favours the side the test was easier for, so it is not counted (Vaidya et al., 2023).
- In an artificial-language study of spacing, three LLMs (including DeepSeek-V3 and Llama 3.1) outperformed people but, unlike people, did better with examples placed side by side than spaced apart, and effects varied by model; because the examples stayed in the prompt, the models' higher scores and missing spacing benefit reflect a test with nothing to forget, not a comparison of forgetting (Xiong & Wu, 2025).
- A 2025 preprint fitted the ACT-R decay parameter to in-context retention curves: four models ranged from d = 0.14 (Llama 3) to 0.35, all forgetting more slowly than a human reference of d = 0.50 (SD 0.05) taken from past modelling work, with Mamba and RWKV-7 closest in overall pattern; earlier material stayed in the context window and no human sample was tested, so this does not count either (Kang et al., 2025).
- The first-pass rating rested on a 2026 CogSci study of the fan effect in 112 people and retrieval systems; that measures interference between competing memories, a separate mechanism, so it is not used here (Guo et al., 2026).
- Comparison Not Possible: every human-AI test of this ability either let the AI keep looking at the material and found it did as well or better than people, or had no documented human sample, and no study places a model's training-based retention (where it cannot look back) against human retention curves or the environmental need statistics people track (Vaidya et al., 2023; Xiong & Wu, 2025; Kang et al., 2025).
Where a Mechanism Can Live
04 Product
The whole configuration, and the only thing a benchmark observes directly.
01
Weights
Across deployments
02
Context Window
Within a session
03
Memory Layer
Across sessions
Which of the three a result belongs to is inferred from it, not seen in it.
Read more
Cerebral cortex
Division in the cerebral hemisphere
The folded sheet of grey matter covering each hemisphere, two to four millimetres thick and layered rather than uniform.
The cortex is a continuous sheet of neurons and local circuitry spread over the hemisphere's surface, thrown into gyri and sulci so that roughly two-thirds of it lies buried in the folds. Most of it (the neocortex) has six layers distinguished by cell type and by which connections arrive and leave; older allocortex has three or four. Areas differ in layer thickness, cell packing and connectivity, which is what cytoarchitectonic maps such as Brodmann's record. Where any given area's boundary actually falls, and how closely those maps correspond to functional divisions, is still argued over; modern multimodal parcellations disagree with Brodmann in places.
Dentate gyrus
Cortex in the hippocampal formation
The input stage of the hippocampus, where entorhinal signals are recoded into sparse, strongly separated patterns of granule cell activity.
A V-shaped band of densely packed granule cells capping CA3, receiving the perforant path from entorhinal cortex and sending unusually large mossy fiber synapses onward. Granule cells fire rarely and in small numbers, which is the anatomical basis for treating the dentate as a pattern separator: making similar inputs into dissimilar codes so they do not overwrite one another. It is also one of two sites where adult neurogenesis is claimed in humans: a live dispute, with methodologically careful studies published on both sides within the same year.
Hippocampus
Cortex in the hippocampal formation
A curved allocortical structure in the medial temporal lobe whose bilateral damage reliably produces dense anterograde amnesia in humans.
A paired, seahorse-shaped fold of three-layered cortex running front to back inside the temporal lobe, connected to the rest of the brain mainly through entorhinal cortex and the fornix. What it does is genuinely contested and has been for fifty years: the human lesion literature reads it as necessary for forming new declarative memories, the rodent recording literature as a map of space, and a third position treats both as instances of binding arbitrary elements into relations. The accounts differ on which species and which task they explain best, and the question of which is right remains open.
Prefrontal cortex
Cortex in the frontal lobe
The frontal cortex in front of the motor areas, defined classically by its connections with the mediodorsal thalamus rather than by any one function.
The prefrontal cortex occupies the front of the frontal lobe, anterior to premotor and motor cortex, and accounts for roughly a sixth to a quarter of human cortex depending on where the posterior border is drawn. Three competing definitions are in use (projection from the mediodorsal thalamus, the presence of a granular layer IV, and the absence of movement on electrical stimulation), and they do not enclose the same tissue, which is why rodent-to-primate homology is still argued over. It is reliably engaged by tasks with delays, rules, and conflicting options; what it contributes to those tasks is the open question, not whether it participates.
Visual cortex
Cortex in the occipital lobe
The occipital cortex processing vision, from the primary area in the calcarine sulcus outward through numbered extrastriate areas.
Visual cortex fills most of the occipital lobe and extends forward into the temporal and parietal lobes. The primary area lies in and around the calcarine sulcus; beyond it, a series of areas (V2, V3, V4, V5/MT and more) each carry a map of the visual field, identified by the reversals of that map rather than by any visible landmark. The division into a dorsal stream for spatial and visuomotor processing and a ventral stream for object recognition, proposed by Ungerleider and Mishkin and revised by Milner and Goodale, remains the organizing frame and remains argued about at its edges.
Anderson & Schooler (1991)
Reflections of the Environment in Memory
In
Psychological Science, 2(6), 396-408
Abstract
Availability of human memories for specific items shows reliable relationships to frequency, recency, and pattern of prior exposures to the item. These relationships have defied a systematic theoretical treatment. A number of environmental sources (New York Times, parental speech, electronic mail) are examined to show that the probability that a memory will be needed also shows reliable relationships to frequency, recency, and pattern of prior exposures. Moreover, the environmental relationships are the same as the memory relationships. It is argued that human memory has the form it does because it is adapted to these environmental relationships. Models for both the environment and human...
Find it
Anderson & Hulbert (2021)
Active Forgetting: Adaptation of Memory by Prefrontal Control
In
Annual Review of Psychology, 72(1), 1-36
Abstract
Over the past century, psychologists have discussed whether forgetting might arise from active mechanisms that promote memory loss to achieve various functions, such as minimizing errors, facilitating learning, or regulating one's emotional state. The past decade has witnessed a great expansion in knowledge about the brain mechanisms underlying active forgetting in its varying forms. A core discovery concerns the role of the prefrontal cortex in exerting top-down control over mnemonic activity in the hippocampus and other brain structures, often via inhibitory control. New findings reveal that such processes not only induce forgetting of specific memories but also can suppress the...
Find it
Chang et al. (2024)
How Do Large Language Models Acquire Factual Knowledge During Pretraining?
In
arXiv
Abstract
Despite the recent observation that large language models (LLMs) can store substantial factual knowledge, there is a limited understanding of the mechanisms of how they acquire factual knowledge through pretraining. This work addresses this gap by studying how LLMs acquire factual knowledge during pretraining. The findings reveal several important insights into the dynamics of factual knowledge acquisition during pretraining. First, counterintuitively, we observe that pretraining on more data shows no significant improvement in the model's capability to acquire and maintain factual knowledge. Next, there is a power-law relationship between training steps and forgetting of memorization and...
Find it
Guo et al. (2026)
The Interference Gap: Comparing Retrieval Bounds in Human Memory and RAG Systems
In
arXiv
Abstract
How do retrieval bounds compare between human episodic memory and Retrieval-Augmented Generation (RAG) systems under semantic interference? We present a unified signal detection theory (SDT) framework that applies to both, and use it to fit behavioral and computational data in matched paradigms. Both systems show logarithmic accuracy decline with association count (fan), but humans exhibit lower interference sensitivity ($α/σ= 0.41$) than dense passage retrieval ($α/σ= 0.67$), with cognitively-inspired HippoRAG falling between the two ($α/σ= 0.44$). Behavioral experiments ($N = 112$) and simulations validate the framework; parameter recovery confirms identifiability ($r \geq .93$) and...
Find it
Jagielski et al. (2023)
Measuring Forgetting of Memorized Training Examples
In
arXiv
Abstract
Machine learning models exhibit two seemingly contradictory phenomena: training data memorization, and various forms of forgetting. In memorization, models overfit specific training examples and become susceptible to privacy attacks. In forgetting, examples which appeared early in training are forgotten by the end. In this work, we connect these phenomena. We propose a technique to measure to what extent models "forget" the specifics of training examples, becoming less susceptible to privacy attacks on examples they have not seen recently. We show that, while non-convex models can memorize data forever in the worst-case, standard image, speech, and language models empirically do forget...
Find it
Janik (2023)
Aspects of human memory and Large Language Models
In
arXiv
Abstract
Large Language Models (LLMs) are huge artificial neural networks which primarily serve to generate text, but also provide a very sophisticated probabilistic model of language use. Since generating a semantically consistent text requires a form of effective memory, we investigate the memory properties of LLMs and find surprising similarities with key characteristics of human memory. We argue that the human-like memory properties of the Large Language Model do not follow automatically from the LLM architecture but are rather learned from the statistics of the training textual data. These results strongly suggest that the biological features of human memory leave an imprint on the way that...
Find it
Kandpal et al. (2023)
Large Language Models Struggle to Learn Long-Tail Knowledge
In
arXiv
Abstract
The Internet contains a wealth of knowledge -- from the birthdays of historical figures to tutorials on how to code -- all of which may be learned by language models. However, while certain pieces of information are ubiquitous on the web, others appear extremely rarely. In this paper, we study the relationship between the knowledge memorized by large language models and the information in pre-training datasets scraped from the web. In particular, we show that a language model's ability to answer a fact-based question relates to how many documents associated with that question were seen during pre-training. We identify these relevant documents by entity linking pre-training datasets and...
Find it
Kang et al. (2025)
In-Context Learning can Perform Continual Learning Like Humans
In
arXiv
Abstract
Large language models (LLMs) can adapt to new tasks via in-context learning (ICL) without parameter updates, making them powerful learning engines for fast adaptation. While extensive research has examined ICL as a few-shot learner, whether it can achieve long-term retention and cross-task knowledge accumulation when multitasks arrive sequentially remains underexplored. Motivated by human memory studies, we investigate the retention characteristics of ICL in multitask settings and extend it to in-context continual learning (ICCL), where continual learning ability emerges through task scheduling and prompt rearrangement. Experiments on Markov-Chain benchmarks demonstrate that, for specific...
Find it
Liu et al. (2024)
Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-Context Models
In
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 4667-4682
Abstract
Numerous recent works target to extend effective context length for language models and various methods, tasks and benchmarks exist to measure model’s effective memory length. However, through thorough investigations, we find limitations for currently existing evaluations on model’s memory. We provide an extensive survey for limitations in this work and propose a new method called forgetting curve to measure the memorization capability of long-context models. We show that forgetting curve has the advantage of being robust to the tested corpus and the experimental settings, of not relying on prompt and can be applied to any model size. We apply our forgetting curve to a large variety of...
Find it
Mallen et al. (2023)
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
In
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 9802-9822
Abstract
Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the difficulty of encoding a wealth of world knowledge in their parameters. This paper aims to understand LMs’ strengths and limitations in memorizing factual knowledge, by conducting large-scale knowledge probing experiments on two open-domain entity-centric QA datasets: PopQA, our new dataset with 14k questions about long-tail entities, and EntityQuestions, a widely used open-domain QA dataset. We find that LMs struggle with less popular factual knowledge, and that retrieval augmentation helps significantly in these cases. Scaling, on the...
Find it
Potter et al. (2018)
Does inhibition cause forgetting after selective retrieval? A reanalysis and failure to replicate
In
Cortex, 104, 26-45
Abstract
Retrieval practice can produce forgetting, but it remains unclear using only behavioral data whether this forgetting is caused by targeted inhibition versus interference. Therefore, Wimber et al. (2015) used pattern classifier analyses of fMRI data to track individual memories in a novel variant of retrieval induced forgetting. After initial learning, people recalled target images across selective retrieval practice trials, and cortical activity patterns gradually became more similar to those evoked by the target pictures (i.e., pattern enhancement) and less similar to those evoked by competing pictures (i.e., pattern suppression). The key question was whether this inhibition of competing...
Find it
Richards & Frankland (2017)
The Persistence and Transience of Memory
In
Neuron, 94(6), 1071-1084
Abstract
The predominant focus in the neurobiological study of memory has been on remembering (persistence). However, recent studies have considered the neurobiology of forgetting (transience). Here we draw parallels between neurobiological and computational mechanisms underlying transience. We propose that it is the interaction between persistence and transience that allows for intelligent decision-making in dynamic, noisy environments. Specifically, we argue that transience (1) enhances flexibility, by reducing the influence of outdated information on memory-guided decision-making, and (2) prevents overfitting to specific past events, thereby promoting generalization. According to this view, the...
Find it
Vaidya et al. (2023)
Humans and language models diverge when predicting repeating text
In
arXiv
Abstract
Language models that are trained on the next-word prediction task have been shown to accurately model human behavior in word prediction and reading speed. In contrast with these findings, we present a scenario in which the performance of humans and LMs diverges. We collected a dataset of human next-word predictions for five stimuli that are formed by repeating spans of text. Human and GPT-2 LM predictions are strongly aligned in the first presentation of a text span, but their performance quickly diverges when memory (or in-context learning) begins to play a role. We traced the cause of this divergence to specific attention heads in a middle layer. Adding a power-law recency bias to these...
Find it
Xiong & Wu (2025)
Do large language models learn like humans: Interleaved and spaced practice in morphological learning
In
Acta Psychologica, 260, 105518
In summary
The study compares how humans and LLMs acquire morphological patterns in an artificial language, varying blocked versus interleaved sequencing and juxtaposed versus spaced presentation across verb classification and inflection tasks. The three models tested consistently outperformed humans, showing superior few-shot learning, but diverged from human cognition in three ways. Humans benefited more from interleaving under juxtaposition, consistent with the discriminative contrast hypothesis, while model behaviour was architecture-dependent. Humans benefited from spaced exemplars while models performed better with juxtaposed input. Humans showed a metacognitive illusion, preferring blocked...
Find it
Zhong et al. (2023)
MemoryBank: Enhancing Large Language Models with Long-Term Memory
In
Proceedings of the AAAI Conference on Artificial Intelligence, 38(17), 19724-19731
Abstract
Large Language Models (LLMs) have drastically reshaped our interactions with artificial intelligence (AI) systems, showcasing impressive performance across an extensive array of tasks. Despite this, a notable hindrance remains—the deficiency of a long-term memory mechanism within these models. This shortfall becomes increasingly evident in situations demanding sustained interaction, such as personal companion systems, psychological counseling, and secretarial assistance. Recognizing the necessity for long-term memory, we propose MemoryBank, a novel memory mechanism tailored for LLMs. MemoryBank enables the models to summon relevant memories, continually evolve through continuous memory...
Find it