Mechanisms
Memory Processes · Acquisition and Transformation
Integrative Inference
Example
You learn that Marco works in the county hospital's pediatric wing. Separately, you read that the hospital is closing that wing. You start thinking about whether he'll need a new job.
Integrative Inference in Humans
Definition
Integrative inference is the combination of separately encoded facts into a conclusion that neither fact contains on its own. Its defining property is that the conclusion was never directly experienced: it exists only as a relationship between two stored memories that share a common element (Zeithamova, Dominick, & Preston, 2012). The standard laboratory version presents overlapping pairs, one item paired with a second and the second later paired with a third, and then tests whether the person knows the relationship between the first and third items, which were never seen together (Zeithamova, Schlichting, & Preston, 2012). Two things distinguish it from neighbouring abilities. It differs from ordinary recall because the answer cannot be read off any single stored trace, and it differs from reasoning in general because the premises must be supplied from memory rather than from material available in front of the person at the time of judgment (Zeithamova, Schlichting, & Preston, 2012).
Boundaries: The timing of the combination is contested. On the integrative encoding account, combination happens when the second, overlapping event is learned: the shared element reactivates the earlier memory and the new information is encoded into it, so the inference is already stored by test time and can simply be retrieved (Zeithamova, Dominick, & Preston, 2012). On the retrieval-based account, the two memories stay separate and are recalled and recombined only when the novel question is put (Zeithamova, Schlichting, & Preston, 2012). The evidence does not settle this. Some studies find hippocampal engagement specific to inference at test, others find it only during encoding, and the review that catalogues both calls the discrepancy a direct contradiction, suggesting that which mechanism is recruited depends on task features such as how many repetitions the premise pairs receive and whether learning is interleaved or single-trial (Zeithamova, Schlichting, & Preston, 2012). This row is scoped to the functional outcome, meaning the combination of two separately held facts into a third, without committing to when the combination occurs. That scoping is what makes the AI comparison below possible, because the AI mechanisms differ from each other on exactly this axis.
Neural Substrate
Human functional imaging locates the function in a hippocampal and ventromedial prefrontal circuit. In 26 healthy adults learning overlapping image pairs, a classifier trained to distinguish object from scene processing detected reactivation of the third, unseen member of each triad during repeated encoding of the first pair, and the magnitude of that reactivation predicted later inference performance across participants (r = 0.46, p = 0.02). Learning-related decreases in hippocampal activation and learning-related increases in ventromedial prefrontal activation each correlated with inference accuracy (r = 0.51, p = 0.008 and r = 0.38, p = 0.05), and the ventromedial prefrontal relationship survived controlling for memory of the directly trained pairs (partial r = 0.53, p = 0.007). Functional connectivity between hippocampus and ventromedial prefrontal cortex increased across repetitions of overlapping events within each run (F(1,21) = 9.78, p = 0.005). Activation changes in anterior medial temporal lobe cortex, but not in hippocampus itself, tracked the reactivation measure (r = 0.54, p = 0.004), which the authors read as hippocampus driving reactivation indirectly through that region (Zeithamova, Dominick, & Preston, 2012).
Causal evidence in humans is available for the prefrontal component. Five patients with focal, stable, bilateral ventromedial prefrontal lesions and ten age-matched and education-matched comparison participants studied overlapping pairs and were then tested on both the studied pairs and the untrained inference pairs. The lesion group showed no deficit on any studied pair type but was significantly impaired on the inference pairs (U = 43.00, d = 1.39, p = 0.028), and the impairment held when the analysis was restricted to triads where the patient had correctly recalled both component pairs, ruling out a simple memory explanation (Spalding et al., 2018). Equivalent causal evidence for hippocampus is weaker here for a structural reason rather than an empirical one: hippocampal damage impairs direct associative memory as well, which makes the specific contribution to inference hard to isolate, though human patients with hippocampal atrophy do show intact acquisition of trained associations alongside impaired inference in a related paradigm (Zeithamova, Schlichting, & Preston, 2012). Note that the sample in the lesion study is five patients, which is typical for focal-lesion work but leaves the effect size imprecisely estimated.
AI at a Glance
Human memory joins separately encoded facts into conclusions neither one contains, using a hippocampal and ventromedial prefrontal circuit that lesion evidence shows to be necessary rather than incidental (Spalding et al., 2018). AI systems perform the same join well below human level when both facts sit in the context window (Maharana et al., 2024), and unevenly when the facts have to be composed out of the weights instead, with performance swinging from over 80% to about 6% depending on the type of the linking item (Yang et al., 2025).
Integrative Inference in AI
How Far It Has Been Measured
Measurement in AI: Directly Measured
AI Memory Performance: Not Yet Compared
Where It Lives in the System
Weights; Product level (Context + Memory layer)
What Has Been Measured
- Transitive inference has been tested by name in large language models given the premises in the conversation (in the model's context window, meaning what it can see at the moment): GPT-3.5-Turbo, GPT-4, Llama3-8B and Qwen showed strong inference on 10-item hierarchies with human-like symbolic distance and terminal-item effects, but the abstract reports no human group and no accuracy figures (Wu & Deng, 2025).
- The systems differed in how they responded to help: step-by-step (chain-of-thought) and confidence prompts improved GPT-4 on transitive inference but lowered accuracy in the other models tested, and terminal-item biases persisted or grew (Wu & Deng, 2025).
- A preprint gave small transformers the classic animal and human training procedure (adjacent pairs only): when pairs were learned into the model's weights (its long-term, trained-in knowledge) the models inferred non-adjacent pairs above chance with a symbolic distance effect, but when pairs were only shown in context they showed no transitive inference unless specially pre-trained; in all but the smallest of seven Gemini and Gemma models a 'number line' prompt improved inference over a circle prompt (Geerts et al., 2025).
- For facts learned in training, a preprint fine-tuned Llama 3 8B, Qwen 2.5 7B, GPT-4o-mini and GPT-4o on invented facts placed in separate documents: all models answered single facts accurately but were at chance on questions linking two facts unless allowed to reason step by step in writing, when accuracy became high; in Llama 3 8B, seeing both facts together in one document or in the conversation gave non-zero accuracy (Balesni, Korbak, & Evans, 2024).
- Using real facts from pretraining and excluding questions whose answer could have been memorized directly, the best models linked two facts without writing out intermediate steps 80% of the time when the linking item was a country but only 5% of the time when it was a year, a wide spread across fact types (Yang et al., 2025).
- For saved memory, a long-conversation benchmark asked questions that require combining information from different sessions: across five systems multi-hop scores ranged from 12.8 to 42.0 F1 (GPT-3.5-turbo-16K best) against a listed human score of 85.8, but the paper does not report how many people answered or whether they could reread the conversations (Maharana et al., 2024).
- Not yet compared: no study gives people and AI the same integrative inference test, though one could; the only human score beside AI (Maharana et al., 2024) lacks sample and procedure; in-context tests let AI see the premises while people must remember them, so strong AI results there cannot count; and training-based failures come from a no-written-reasoning condition people are not held to, with no human data on a shared scale (Balesni, Korbak, & Evans, 2024).
Where a Mechanism Can Live
04 Product
The whole configuration, and the only thing a benchmark observes directly.
01
Weights
Across deployments
02
Context Window
Within a session
03
Memory Layer
Across sessions
Which of the three a result belongs to is inferred from it, not seen in it.
Read more
Hippocampus
Cortex in the hippocampal formation
A curved allocortical structure in the medial temporal lobe whose bilateral damage reliably produces dense anterograde amnesia in humans.
A paired, seahorse-shaped fold of three-layered cortex running front to back inside the temporal lobe, connected to the rest of the brain mainly through entorhinal cortex and the fornix. What it does is genuinely contested and has been for fifty years: the human lesion literature reads it as necessary for forming new declarative memories, the rodent recording literature as a map of space, and a third position treats both as instances of binding arbitrary elements into relations. The accounts differ on which species and which task they explain best, and the question of which is right remains open.
Medial temporal lobe
Cortex in the temporal lobe
The inner fold of the temporal lobe (hippocampus, the cortices wrapped around it, and the amygdala), removed together in the classic amnesia cases.
A regional grouping rather than a single structure: the hippocampal formation, the entorhinal, perirhinal and parahippocampal cortices that feed it, and, on some definitions, the amygdala. It became the canonical "memory region" because the surgeries and infarcts that produce dense anterograde amnesia (H.M. most famously) take out most of it at once. Whether the parts constitute one memory system or several with different jobs is an open argument, and lesions in patients often cross the boundaries that would settle it.
Prefrontal cortex
Cortex in the frontal lobe
The frontal cortex in front of the motor areas, defined classically by its connections with the mediodorsal thalamus rather than by any one function.
The prefrontal cortex occupies the front of the frontal lobe, anterior to premotor and motor cortex, and accounts for roughly a sixth to a quarter of human cortex depending on where the posterior border is drawn. Three competing definitions are in use (projection from the mediodorsal thalamus, the presence of a granular layer IV, and the absence of movement on electrical stimulation), and they do not enclose the same tissue, which is why rodent-to-primate homology is still argued over. It is reliably engaged by tasks with delays, rules, and conflicting options; what it contributes to those tasks is the open question, not whether it participates.
Ventromedial prefrontal cortex
Cortex in the medial prefrontal cortex
The lower medial corner of the prefrontal cortex, where the medial wall meets the orbital surface above the eye sockets.
The ventromedial prefrontal cortex occupies the junction of the medial and orbital surfaces of the frontal lobe, covering parts of Brodmann areas 10, 11, 14, 25 and 32; its overlap with medial orbitofrontal cortex is substantial and inconsistently labeled. It is heavily connected with the amygdala, ventral striatum, and hypothalamus. Damage here (the Phineas Gage and Elliot cases, and later lesion series) produces poor real-world decisions with intact intelligence, and the region is central to the somatic marker hypothesis. Its popular description as a moral-decision region is a strong reading of weak evidence: the same tissue tracks subjective value, regulation of fear responses, and memory schemas.
Balesni et al. (2024)
Lessons from Studying Two-Hop Latent Reasoning
In
arXiv
Abstract
Large language models can use chain-of-thought (CoT) to externalize reasoning, potentially enabling oversight of capable LLM agents. Prior work has shown that models struggle at two-hop question-answering without CoT. This capability is so basic that if it was a fundamental limitation, it would imply that many complex agentic tasks would similarly require CoT. We investigate LLM latent reasoning capabilities using two-hop question answering as a case study. Previous work on the gap between latent and externalized two-hop reasoning produced mixed evidence with inconclusive results. In this paper, we introduce a controlled setting for investigating two-hop reasoning in LLMs, where a...
Find it
Geerts et al. (2025)
Relational reasoning and inductive bias in transformers and large language models
In
arXiv
Abstract
Transformer-based models have demonstrated remarkable reasoning abilities, but the mechanisms underlying relational reasoning remain poorly understood. We investigate how transformers perform \textit{transitive inference}, a classic relational reasoning behavior from psychology which elicits inference about indirectly related items (e.g., if $A > B$ and $B > C$, then $A > C$). We compare in-weights learning (IWL) and in-context learning (ICL) behaviors and mechanisms on these tasks, and fine profoundly different patterns of generalization. IWL models learn a linear embedding, which leads to transitive inference as well as other behavioral effects present in humans and animals. ICL models,...
Find it
Maharana et al. (2024)
Evaluating Very Long-Term Conversational Memory of LLM Agents
In
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 13851-13870
Abstract
Paraphrase, not the publisher's wording. Prior work on long-term open-domain dialogue evaluates model responses over no more than five chat sessions, leaving the effectiveness of long-context models and retrieval augmentation in genuinely long-term conversation untested. The authors introduce a machine-human pipeline that generates high-quality very long-term dialogues by grounding LLM agent conversations in personas and temporal event graphs, with agents able to share and react to images. Human annotators verify and edit the output for long-range consistency and grounding. The resulting dataset, LoCoMo, supports an evaluation benchmark covering question answering, event summarisation,...
Find it
Spalding et al. (2018)
Ventromedial Prefrontal Cortex Is Necessary for Normal Associative Inference and Memory Integration
In
The Journal of Neuroscience, 38(15), 3767-3775
Abstract
The ability to flexibly combine existing knowledge in response to novel circumstances is highly adaptive. However, the neural correlates of flexible associative inference are not well characterized. Laboratory tests of associative inference have measured memory for overlapping pairs of studied items (e.g., AB, BC) and for nonstudied pairs with common associates (i.e., AC). Findings from functional neuroimaging and neuropsychology suggest the ventromedial prefrontal cortex (vmPFC) may be necessary for associative inference. Here, we used a neuropsychological approach to test the necessity of vmPFC for successful memory-guided associative inference in humans using an overlapping pairs...
Find it
Wu & Deng (2025)
Transitive Inference in Large Language Models and Prompting Intervention
In
ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1-5
Find it
Yang et al. (2025)
Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?
In
Findings of the Association for Computational Linguistics: ACL 2025, 3971-3992
Abstract
We evaluate how well Large Language Models (LLMs) latently recall and compose facts to answer multi-hop queries like “In the year Scarlett Johansson was born, the Summer Olympics were hosted in the country of”. One major challenge in such evaluation is that LLMs may have developed shortcuts by encountering the head entity “Scarlett Johansson” and the answer entity “United States” in the same training sequences or merely guess the answer based on frequency-based priors. To prevent shortcuts, we exclude test queries where the head and answer entities might have co-appeared during training. Through careful selection of relations and facts and systematic removal of cases where models might...
Find it
Zeithamova et al. (2012)
Hippocampal and Ventral Medial Prefrontal Activation during Retrieval-Mediated Learning Supports Novel Inference
In
Neuron, 75(1), 168-179
Abstract
Memory enables flexible use of past experience to inform new behaviors. Although leading theories hypothesize that this fundamental flexibility results from the formation of integrated memory networks relating multiple experiences, the neural mechanisms that support memory integration are not well understood. Here, we demonstrate that retrieval-mediated learning, whereby prior event details are reinstated during encoding of related experiences, supports participants' ability to infer relationships between distinct events that share content. Furthermore, we show that activation changes in a functionally coupled hippocampal and ventral medial prefrontal cortical circuit track the formation...
Find it
Zeithamova et al. (2012)
The hippocampus and inferential reasoning: building memories to navigate future decisions
In
Frontiers in Human Neuroscience, 6
Abstract
A critical aspect of inferential reasoning is the ability to form relationships between items or events that were not experienced together. This review considers different perspectives on the role of the hippocampus in successful inferential reasoning during both memory encoding and retrieval. Intuitively, inference can be thought of as a logical process by which elements of individual existing memories are retrieved and recombined to answer novel questions. Such flexible retrieval is sub-served by the hippocampus and is thought to require specialized hippocampal encoding mechanisms that discretely code events such that event elements are individually accessible from memory. In addition...
Find it