Pink et al. (2026)

Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models

arXiv

Find It

DOI

Abstract

Human episodic memory supports the retrieval of experiences that unfold over extended timescales, yet the computational mechanisms underlying this ability remain debated due to the limited mechanistic accessibility in long-term memory experiments in humans. Long-context LLMs may offer promising ways to reveal plausible computational mechanisms that drive this type of retrieval. Here, we investigate whether and how LLMs capture the core behavioral signatures of episodic memory via a temporal order memory task. Using a new dataset of human behavior based on memory of a full-length novel, we show that models exhibit the same characteristic distance effect observed in humans on this task. We next apply long-context mechanistic interpretability analyses to uncover how models solve this task, and find that model performance relies on a one-dimensional temporal code that is reinstated during retrieval by a single time-reinstatement attention head. These findings support temporal context reinstatement as an important mechanism for episodic-like temporal-order memory in LLMs, offering new insights into how temporal aspects of long-term episodic memory may be instantiated in both artificial and biological systems.

Reference

Pink, M., Vo, V. A., Wu, Q., Mu, J., Turek, J., Hasson, U., Norman, K. A., Michelmann, S., Huth, A., & Toneva, M. (2026). Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models. arXiv preprint arXiv:2607.22575. https://doi.org/10.48550/arxiv.2607.22575

What Rests on This

2 mechanisms cite this source. Each finding below is quoted as it appears on that mechanism's own page.

Temporal Placement Memory Processes

Measured in AI

An ICML 2026 paper gave the same 296 passage pairs to 97 readers (tested within 30 days of finishing the novel) and to models holding the whole novel in their context window, meaning the text they can see in the current conversation. Both showed the human distance effect; Llama 3.1 8B did not differ from readers at any distance (all p ≥ .289), and Llama 3.1 70B (both 2024 releases) beat them only for passages 16,000–24,000 words apart (about 92% versus 72%)

Context-Dependent Retrieval Memory Processes

Measured in AI

A 2026 preprint gave Llama 3.1 8B and 70B a whole novel in context and 97 readers the same 296 which-came-first judgments from memory; both showed the human distance effect, 8B did not differ from people and 70B beat them at one distance range, and one 'time-reinstatement' attention head drove performance (ablation cost 10-15%), but it tests memory for the order of events rather than retrieval helped by a matching context, so it is not used for this rating