Mechanisms
Metacognition · Control
Metacognitive Control
Example
You are asked at dinner who directed a film you saw years ago. The name does not arrive, but something tells you it is close, so you keep hunting for it instead of giving up, cycling through the actors and the year. Thirty seconds later you still do not have it, and now you make a second decision: you could name the director you half suspect, or you could say "one of the New Hollywood people, I want to say early seventies" and be safe, or you could say you cannot remember. Nothing in that sequence is a judgment about your memory. Each step is an action taken because of one: keep searching, stop searching, report at this grain, or withhold.
Metacognitive Control in Humans
Definition
Metacognitive control is the arm of metamemory that acts on memory rather than reading it: it takes whatever the monitoring system reports about a memory's state and converts that report into a decision about what the memory system does next (Nelson & Narens, 1990). The control operations are conventionally grouped by the stage they act on. At acquisition they include choosing what to study, allocating study time across items, and deciding when to stop studying a given item (Nelson & Narens, 1990; Metcalfe & Kornell, 2005). At retrieval they include deciding whether to initiate a search at all, deciding when to abandon a search that is not converging, deciding whether to volunteer the retrieved answer or withhold it, and deciding how precisely to state it, a choice known as grain-size control (Nelson & Narens, 1990; Koriat & Goldsmith, 1996; Goldsmith, Koriat, & Weinberg-Eliezer, 2002). The report decision is the most extensively modelled of these: the rememberer compares the assessed probability that a candidate answer is correct against a response criterion set by the relative payoffs for being informative and for being right, volunteers the answer if the assessment clears the criterion, and withholds it otherwise (Koriat & Goldsmith, 1996). Control is therefore defined by its output, an action on the memory system, which is what separates it from monitoring, defined by its output, a judgment about the memory system.
Scope note: This row covers the control arm only. Confidence, calibration, and monitoring resolution are treated in the Confidence row, and source judgments in the Attribution row, even though control cannot be evaluated without reference to the monitoring signal it reads. Two boundaries are contested in the source literature. First, whether every monitoring-and-control loop counts as metacognitive: performance adjustments can occur without any reportable awareness of them, and Fleming and Dolan (2012) argue for restricting the term to control that operates on consciously reportable representations rather than to closed-loop performance regulation generally. Second, whether monitoring and control are separable processes at all, on which the neural sources reviewed below do not agree. Of the six operations named in the definition, the AI-side material below covers report or withhold and search initiation; effort allocation and grain-size control are stated but not evaluated.
Neural Substrate
The clearest human evidence that control is neurally distinguishable from monitoring comes from Boldt and Gilbert (2022), an fMRI study of 21 participants (20 in the multivariate analyses) using a delayed-intention task in which the control act was requesting a reminder, that is, offloading the intention onto an external cue. In separate blocks participants either rated their confidence that they would remember a target (monitoring) or rated how much they wanted a reminder (control). Multivariate pattern analysis decoded high versus low ratings within each condition, and also cross-classified between them, meaning that a classifier trained on one judgment could predict the other, which succeeded above chance in one direction only, training on control and testing on monitoring, in left superior and middle frontal gyri. Within-condition decoding accuracy exceeded cross-condition accuracy, F(1,19) = 6.2, p = 0.02, partial eta squared = 0.25, which the authors read as partial but incomplete overlap. All seven task-responsive regions showed higher univariate signal for the control condition than the monitoring condition, F(1,20) = 8.1, p = 0.01, partial eta squared = 0.29, consistent with control integrating inputs beyond the monitoring signal itself.
Qiu et al. (2018) reach a stronger separation using a decision-redecision paradigm across a rule-based task (Sudoku) and a perceptual task (random-dot motion), with 21 participants in the main experiment and three further fMRI experiments. Dorsal anterior cingulate and anterior insula activity scaled with decision uncertainty and predicted individual uncertainty sensitivity (r = 0.79 Sudoku, r = 0.55 random-dot motion), while lateral frontopolar cortex activity scaled instead with the reduction in uncertainty achieved by redeciding and predicted individual accuracy change (r = 0.69 Sudoku, r = -0.39 random-dot motion), and did not predict uncertainty sensitivity. Two features of that result bear on the construct rather than the anatomy. The frontopolar correlation reverses sign between the two tasks, and individual accuracy change was not consistent across tasks (Cronbach's alpha = 0.03) while individual uncertainty sensitivity was highly consistent (alpha = 0.85). On this evidence, monitoring ability looks like a stable trait and control ability looks task specific.
The sources disagree on separability: Boldt and Gilbert report partially shared representational structure, Qiu et al. argue for a dedicated control system in lateral frontopolar cortex, and Fleming and Dolan (2012), reviewing the earlier literature, place lateral prefrontal cortex downstream of cingulate and insular monitoring signals rather than parallel to them. The disproportion in the evidence base should be stated plainly. Vaccaro and Fleming's (2018) coordinate-based meta-analysis of 47 neuroimaging studies (88 contrasts, 739 foci, 2,215 participants) maps a domain-general network for metacognitive judgments, but its corpus consists almost entirely of judgment studies, and its one analysis bearing on how well metacognition tracks performance drew on 11 studies and produced no clusters surviving correction. The evidence reviewed here is human throughout, but the neural literature on control is small relative to the literature on monitoring.
AI at a Glance
In humans, metacognitive control is the arm of metamemory that converts a monitoring signal into an action on memory, deciding whether to search, how long to study, whether to volunteer an answer or withhold it, and how precisely to state it, with the report decision set by a confidence criterion that shifts with the cost of being wrong (Nelson & Narens, 1990; Koriat & Goldsmith, 1996). AI systems have a directly measured version of the report decision and an indirectly measured version of the search decision, but they withhold unreliably, do not improve with scale, and lose abstention after reasoning fine-tuning, with the criterion in practice set by post-training, the system prompt, and a retrieval threshold rather than by anything the system computes about the state of its own memory (Kirichenko et al., 2025; Wei et al., 2024; Zhang, Fang, & Chen, 2024; Asai et al., 2024).
Metacognitive Control in AI
How Far It Has Been Measured
Measurement in AI: Directly Measured
AI Memory Performance: Below Human approximate provisional
Where It Lives in the System
Weights Context Memory layer Product level
Weights carry the default report policy. Post-training determines how readily a model withholds, and interventions at this level move it substantially: reasoning fine-tuning lowers abstention, and Self-RAG trains a search decision directly into the model's output vocabulary as a special token (Kirichenko et al., 2025; Asai et al., 2024). Context carries the criterion setting at run time. A system prompt describing when to decline shifts abstention up, and an instruction not to decline shifts it down, without touching the weights (Kirichenko et al., 2025; Epoch AI, 2026). The memory layer carries search initiation: the decision whether to query an external store, and the threshold that decision is compared against, live in the retrieval pipeline rather than in the model (Asai et al., 2024; Zhang, Fang, & Chen, 2024). Product level carries two things. The first is the scoring rule under which the model is evaluated and optimised, which in most current benchmarks awards nothing for declining to answer (Kalai, Nachum, Vempala, & Zhang, 2025). The second is the shipped effort-allocation control: user-facing reasoning-effort settings such as the o1 API's reasoning_effort parameter are a deployed component that allocates processing to a query, functionally matching study-time allocation, and no published work evaluates them as metacognitive control, which is the basis for the Unmeasured state above (Kirichenko et al., 2025).
What Has Been Measured
- A peer-reviewed 2026 study tested this ability by name in 17 large language models (AI systems trained on vast amounts of text) using an answer-or-delegate game modelled on animal opt-out tasks, drawing on what the models learned in training; the link between being wrong and choosing to pass was weak (partial correlation 0.3 at best), and only one model, Claude 3.5 Sonnet, used passing to beat both its own and its teammate's accuracy (by 5.4 and 8.6 points) (Ackerman, 2026).
- A 2026 preprint found that four models (GPT-4o, Gemma 3 27B, DeepSeek, Qwen) decide to abstain on trivia questions using internal confidence and a threshold, as in the two-stage framework from animal metacognition; shifting confidence by directly altering the model's internal activity moved abstention from 66.5% to 7.0%, but no human data were collected (Kumaran et al., 2026).
- Across 20 frontier models and 20 datasets of unanswerable, underspecified or outdated questions, knowing when not to answer remained unsolved: larger models were of little help, and training for step-by-step reasoning reduced abstention by 24% on average (Kirichenko et al., 2025).
- For the current conversation and saved memory, withholding was tested with unanswerable trick questions about very long chats: scores (F1, out of 100) ranged from 2.5 (Claude 3 Sonnet) and 15.7 (GPT-4 Turbo) to 80.0 (Llama 3 70B, which could see only a short part of the chat), and from 9.9 to 32.5 when GPT-3.5 looked up stored chat excerpts; a human score of 89.4 is reported without saying how it was obtained, so it cannot be used (Maharana et al., 2024).
- The only documented human comparison found used the SelfAware set, which scores whether a model answers or declines unanswerable questions: GPT-4, the best of 20 models, reached F1 75.5, below both human volunteers (84.4 and 85.5), and other models scored lower, down to about 29 (Yin et al., 2023).
- This comparison is approximate: only two people each answered a random 100-question sample in 30 minutes, the samples may not match GPT-4's, and the questions have no answer at all rather than one the person failed to recall; it still points one way, since every model fell at least 8.9 F1 points below both people and the paper describes no outside help for either side (Yin et al., 2023).
- Same-task human–AI studies so far measure monitoring, not control: with 87 adults and GPT-4 on the same 60 quiz questions, confidence separated right from wrong answers less well for GPT-4 (type-2 AUROC 0.63) than for people on average (0.76, SD 0.14), though within the human spread, and no decision to answer or withhold was recorded (Yoshizawa et al., 2026).
Where a Mechanism Can Live
04 Product
The whole configuration, and the only thing a benchmark observes directly.
01
Weights
Across deployments
02
Context Window
Within a session
03
Memory Layer
Across sessions
Which of the three a result belongs to is inferred from it, not seen in it.
Read more
Anterior insula
Cortex in the insula
The front portion of the insula, agranular and closely tied to the anterior cingulate, amygdala and orbitofrontal cortex.
The anterior insula lies in front of the insular central sulcus, comprises three short gyri, and is agranular or dysgranular in structure. With the anterior cingulate it is one of only two cortical regions containing von Economo neurons in humans and great apes, and the two are strongly interconnected. It is activated during interoceptive attention, risk and uncertainty, disgust, empathy for pain, and awareness of one's own heartbeat. Craig's proposal that it houses a hierarchy culminating in subjective awareness is influential and contested; critics point out that the same tissue activates in almost any demanding task, including ones with no bodily content.
Cingulate cortex
Cortex in the cerebral cortex
The long belt of cortex arching over the corpus callosum on the medial wall of each hemisphere.
The cingulate cortex is a collar of cortex (cingulum is Latin for belt) wrapped around the corpus callosum on the medial surface of the hemisphere, separated from the frontal and parietal cortex above it by the cingulate sulcus. It is mesocortex, with a simpler layered structure than neocortex, and it is bordered below by the callosum and the cingulum bundle that carries its long connections. Vogt's four-region model divides it into anterior, midcingulate, posterior, and retrosplenial sectors with distinct architecture and connections; the older two-part anterior/posterior split is still common in imaging papers and does not map onto it cleanly.
Dorsal anterior cingulate cortex
Cortex in the anterior cingulate cortex
The upper-rear part of the anterior cingulate, above the corpus callosum, the site of most conflict and error findings.
The dorsal anterior cingulate cortex lies above the body and genu of the corpus callosum, in Brodmann area 24 and the adjacent area 32; much of what imaging papers call dACC falls in what Vogt's scheme names anterior midcingulate cortex, so the same coordinates carry two labels in the literature. The cingulate motor areas in the cingulate sulcus here project directly to the spinal cord and to other motor areas, unusual for cingulate tissue. It responds to response conflict, to errors, to physical pain, to social rejection, and to effortful choice. Whether one computation explains that range, or the region is simply near a crossroads, is unsettled.
Frontopolar cortex
Cortex in the prefrontal cortex
The most anterior cortex in the brain, the rounded tip of the frontal lobe, corresponding to Brodmann area 10.
The frontopolar cortex is the forward tip of the frontal lobe and the foremost cortex in the human brain, approximately coextensive with Brodmann area 10, which reaches beyond the pole ventromedially and is proportionally larger in humans than in other primates, with an unusually low density of cells per unit volume, leaving more space for connections. Its inputs come almost entirely from other association cortex rather than from sensory areas. It is activated when people hold one goal in mind while pursuing another, evaluate their own performance, and reason about possibilities. Proposals range from relational integration to branching control to prospective memory, and no single account has displaced the others.
Insula
Cortex in the cerebral hemisphere
A triangular expanse of cortex buried inside the Sylvian fissure, covered over by the frontal, parietal and temporal opercula.
The insula is the only cortical lobe with no exposed surface: it sits at the bottom of the Sylvian fissure, hidden by the opercula that grow over it during development, and is divided by its central sulcus into a larger anterior and smaller posterior part. Its architecture grades from agranular at the front to granular at the back. It receives visceral, thermal, nociceptive and gustatory input. Popular accounts name it the seat of interoception, disgust, empathy, or consciousness in turn; the region appears in a large share of published fMRI contrasts, which makes reverse inference from insular activity to any one of those especially unsafe.
Middle frontal gyrus
Cortex in the frontal lobe
The gyrus between the superior and inferior frontal sulci, occupying most of the lateral prefrontal surface.
The middle frontal gyrus lies between the superior and inferior frontal sulci and extends forward to the frontal pole. Much of what is called dorsolateral prefrontal cortex falls on it, along with the frontal eye field near its junction with the precentral sulcus. It is reported across a wide range of imaging tasks, activated during working memory, attentional control and rule-guided behavior. That ubiquity is a caution rather than a finding: the gyrus is large, its sulcal pattern varies considerably between individuals, and group-average coordinates on it can correspond to different cytoarchitectonic areas in different brains.
Prefrontal cortex
Cortex in the frontal lobe
The frontal cortex in front of the motor areas, defined classically by its connections with the mediodorsal thalamus rather than by any one function.
The prefrontal cortex occupies the front of the frontal lobe, anterior to premotor and motor cortex, and accounts for roughly a sixth to a quarter of human cortex depending on where the posterior border is drawn. Three competing definitions are in use (projection from the mediodorsal thalamus, the presence of a granular layer IV, and the absence of movement on electrical stimulation), and they do not enclose the same tissue, which is why rodent-to-primate homology is still argued over. It is reliably engaged by tasks with delays, rules, and conflicting options; what it contributes to those tasks is the open question, not whether it participates.
Ackerman (2026)
Evidence for Limited Metacognition in LLMs
In
arXiv
Abstract
The possibility of LLM self-awareness and even sentience is gaining increasing public attention and has major safety and policy implications, but the science of measuring them is still in a nascent state. Here we introduce a novel methodology for quantitatively evaluating metacognitive abilities in LLMs. Taking inspiration from research on metacognition in nonhuman animals, our approach eschews model self-reports and instead tests to what degree models can strategically deploy knowledge of internal states. Using two experimental paradigms, we demonstrate that frontier LLMs introduced since early 2024 show increasingly strong evidence of certain metacognitive abilities, specifically the...
Find it
Asai et al. (2024)
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
In
arXiv
Abstract
Despite their remarkable capabilities, large language models (LLMs) often produce responses containing factual inaccuracies due to their sole reliance on the parametric knowledge they encapsulate. Retrieval-Augmented Generation (RAG), an ad hoc approach that augments LMs with retrieval of relevant knowledge, decreases such issues. However, indiscriminately retrieving and incorporating a fixed number of retrieved passages, regardless of whether retrieval is necessary, or passages are relevant, diminishes LM versatility or can lead to unhelpful response generation. We introduce a new framework called Self-Reflective Retrieval-Augmented Generation (Self-RAG) that enhances an LM's quality and...
Find it
Kalai & Vempala (2025)
Calibrated Language Models Must Hallucinate
In
Proceedings of the 56th Annual ACM Symposium on Theory of Computing, 160-171
Abstract
Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Such "hallucinations" persist even in state-of-the-art systems and undermine trust. We argue that language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty, and we analyze the statistical causes of hallucinations in the modern training pipeline. Hallucinations need not be mysterious -- they originate simply as errors in binary classification. If incorrect statements cannot be distinguished from facts, then hallucinations in pretrained language models...
Find it
Kirichenko et al. (2025)
AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
In
arXiv
Abstract
For Large Language Models (LLMs) to be reliably deployed in both everyday and high-stakes domains, knowing when not to answer is equally critical as answering correctly. Real-world user queries, which can be underspecified, ill-posed, or fundamentally unanswerable, require LLMs to reason about uncertainty and selectively abstain -- i.e., refuse to answer definitively. However, abstention remains understudied, without a systematic evaluation framework for modern LLMs. In this work, we introduce AbstentionBench, a large-scale benchmark for holistically evaluating abstention across 20 diverse datasets, including questions with unknown answers, underspecification, false premises, subjective...
Find it
Koriat & Goldsmith (1996)
Monitoring and control processes in the strategic regulation of memory accuracy.
In
Psychological Review, 103(3), 490-517
Abstract
When people are allowed freedom to volunteer or withhold information, they can enhance the accuracy of their memory reports substantially relative to forced-report performance. A theoretical framework addressing the strategic regulation of memory reporting is put forward that delineates the mediating role of metamemorial monitoring and control processes. Although the enhancement of memory accuracy is generally accompanied by a reduction in memory quantity, experimental and simulation results indicate that both of these effects depend critically on (a) accuracy incentive and ( b ) monitoring effectiveness. The results are discussed with regard to the contribution of metamemory processes to...
Find it
Maharana et al. (2024)
Evaluating Very Long-Term Conversational Memory of LLM Agents
In
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 13851-13870
Abstract
Paraphrase, not the publisher's wording. Prior work on long-term open-domain dialogue evaluates model responses over no more than five chat sessions, leaving the effectiveness of long-context models and retrieval augmentation in genuinely long-term conversation untested. The authors introduce a machine-human pipeline that generates high-quality very long-term dialogues by grounding LLM agent conversations in personas and temporal event graphs, with agents able to share and react to images. Human annotators verify and edit the output for long-range consistency and grounding. The resulting dataset, LoCoMo, supports an evaluation benchmark covering question answering, event summarisation,...
Find it
Metcalfe & Kornell (2005)
A Region of Proximal Learning model of study time allocation
In
Journal of Memory and Language, 52(4), 463-477
Abstract
A Region of Proximal Learning model is proposed emphasizing two components to people’s study time allocation, controlled by different metacognitive indices. The first component is choice, which is further segmented into two stages: (1) a decision of whether to study or not and (2) the order of priority of items chosen. If the people’s Judgments of Learning (JOLs) are sufficiently high that they believe they know the items already, they will choose to not study. If they do choose to study, the order is from that which they believe is almost known to that which they believe is more difficult (high JOL to low JOL). The second component is perseverance, with emphasis on the rule for stopping...
Find it
Nelson (1990)
Metamemory: A Theoretical Framework and New Findings
In
Psychology of Learning and Motivation, 125-173
Find it
Wei et al. (2024)
Long-form factuality in large language models
In
Advances in Neural Information Processing Systems 37, 80756-80827
Abstract
We present SimpleQA, a benchmark that evaluates the ability of language models to answer short, fact-seeking questions. We prioritized two properties in designing this eval. First, SimpleQA is challenging, as it is adversarially collected against GPT-4 responses. Second, responses are easy to grade, because questions are created such that there exists only a single, indisputable answer. Each answer in SimpleQA is graded as either correct, incorrect, or not attempted. A model with ideal behavior would get as many questions correct as possible while not attempting the questions for which it is not confident it knows the correct answer. SimpleQA is a simple, targeted evaluation for whether...
Find it
Yin et al. (2023)
Do Large Language Models Know What They Don’t Know?
In
Findings of the Association for Computational Linguistics: ACL 2023, 8653-8665
Abstract
Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks. Current research focuses on enhancing their performance within their existing knowledge. Despite their vast knowledge, LLMs are still limited by the amount of information they can accommodate and comprehend. Therefore, the ability to understand their own limitations on the unknows, referred to as self-knowledge, is of paramount importance. This study aims to evaluate LLMs’ self-knowledge by assessing their ability to identify unanswerable or unknowable questions. We introduce an automated methodology to detect uncertainty in the responses of these models,...
Find it
Yoshizawa et al. (2026)
Metacognition of ChatGPT in confidence judgements
In
Frontiers in Artificial Intelligence, 9
Abstract
Recent advances in Large Language Models (LLMs) have raised critical concerns regarding AI alignment and safety, particularly with respect to the reliability of their outputs. In humans, metacognition plays a key role in making cognition robust and adaptive. LLMs frequently express high confidence in their responses, raising the question of whether such confidence reflects human-like metacognitive capability. In this study, we systematically compared humans and GPT-4 across multiple task formats to examine how confidence relates to performance. GPT-4 consistently outperformed humans in task accuracy. This advantage was not accompanied by human-like confidence behavior: Human confidence...
Find it
Zhang et al. (2024)
RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering
In
Findings of the Association for Computational Linguistics ACL 2024, 6963-6975
Abstract
Adaptive retrieval-augmented generation (ARAG) aims to dynamically determine the necessity of retrieval for queries instead of retrieving indiscriminately to enhance the efficiency and relevance of the sourced information. However, previous works largely overlook the evaluation of ARAG approaches, leading to their effectiveness being understudied. This work presents a benchmark, RetrievalQA, comprising 1,271 short-form questions covering new world and long-tail knowledge. The knowledge necessary to answer the questions is absent from LLMs; therefore, external information must be retrieved to answer correctly. This makes RetrievalQA a suitable testbed to evaluate existing ARAG methods. We...
Find it