Scoring

How Each Mechanism Was Scored

The criteria every mechanism is scored against, then the findings behind each score: what the mechanism is, where it lives in the brain, how it has been measured in people, and how it has been measured in AI.

Score audit: the checks run against these scores →

Scoring Criteria

Level of Measurement: Humans

Every mechanism here is established in humans, so the question is only how directly.

Directly Measured
Psychologists have a standard experiment built to test this ability specifically, and its results have been replicated or pooled in a meta-analysis.
Indirectly Measured
The ability is known from other kinds of evidence, such as diary studies, people with brain injuries, or experiments designed to test something else, rather than from a test built for it.

Level of Measurement: AI

An AI system can hold information in three places: what the model learned in training, what it can see in the current conversation, and any saved memory it can look things up in. Each was checked.

Directly Measured
A published study tests AI on this ability by name, or gives AI the same experiment psychologists use to test it in people.
Indirectly Measured
Studies have tested something close to this ability, but under a different name, as part of a broader test, or in only some of the places an AI system holds information.

AI Memory Performance

Rating AI against people takes results from both on the same ability. Those results can be compared in two ways.

Direct comparison
People and AI took the same test, with the same questions, scored the same way, and neither side could look back at the material unless the other could too.
Approximate comparison
People and AI took different tests of the same ability, but the results can be put on a common scale, such as accuracy above chance or a standard effect size. A rating is given only when the result is clear despite the differences between the tests, and it is labelled approximate.

Either way, two conditions apply:

  1. 1 The human results are documented. The study reports how many people were tested and how.
  2. 2 An uneven test only counts against its advantage. When a test was easier for one side, a result that goes against that advantage counts, and so does a tie. An AI that could still see the material and scored below people is rated Below Human. One that could see it and only matched people is rated Comparable, and always marked provisional. One that could see it and scored above people is not counted.

The ratings:

Exceeds Human
AI did better than the normal range for people.
Comparable to Humans
AI fell within the normal range for people.
Below Human
AI did worse than the normal range for people.
Not Yet Compared
Both people and AI have been tested on this ability, and the two could be put on one scale, but no study has yet done so with documented human results. An AI benchmark that prints a human score without saying who was tested or how does not count as that study.
Comparison Not Possible
No fair test could put people and AI on one scale for this ability: what it measures in people has no counterpart that could be tested the same way in AI.

There is no Absent rating. When nothing in an AI system does what an ability does, there is nothing to compare, and the rating is Comparison Not Possible.

The normal range for people
The spread the human studies report: a standard deviation, a confidence interval, or the range across groups. If a study gives only an average, a difference counts only when the researchers' own statistics find one.
Which AI system
When a study tests several AI systems, the best one is rated, since the question is what today's AI memory can do. How much the systems differed is noted alongside.
Provisional
A rating is marked provisional when it rests on one study, on studies not yet peer reviewed, on a single AI system, or on a tie from a test that was easier for the AI.
When results disagree
  1. 1 Any comparison outranks none. A result that can be compared, directly or approximately, takes precedence over Not Yet Compared and Comparison Not Possible, which record that no comparison was made, not that one came out a particular way.
  2. 2 Where comparisons disagree, a direct comparison outranks an approximate one. Between comparisons of the same strength, the one whose test better captures the ability as psychologists study it in people stands. The others are still shown.
  3. 3 Where no test is clearly the better one and the comparisons point in different directions, the more cautious rating stands, meaning the lower one, and it is marked provisional.

Definitions, what each score requires, and worked examples are on the Method page.

Table 1a, Scored Against These Criteria

Where each mechanism lands when scored by the criteria above. Select a dot to jump to that mechanism's findings. Hollow dots are being re-checked.

Measurement in AI

Directly (14)
Indirectly (15)

Not possible (6)

Not yet (9)

Below (8)

Compar­able (5)

Ex­ceeds (1)

not on the scale

lower higher

AI Memory Performance

Level of measurement

Directly
Tested in AI by name, or with the human experiment.
Indirectly
Tested under another name, or only in part.

AI Memory Performance

Comparison Not Possible
No fair test could put both on one scale.
Not Yet Compared
Could be compared; no study has done it yet.
Below Human
AI did worse than people.
Comparable to Humans
AI fell within the range for people.
Exceeds Human
AI did better than people.
  • Memory Systems
  • Representational Structures
  • Memory Processes
  • Metacognition

Each dot is one of the 29 mechanisms. Where a dot sits within a cell carries no meaning, just which cell it is in.

What Each Relationship Needs, Against How AI Measures Up

Every mechanism in one bar, coloured by how AI memory compares with people. Under it, a filled cell marks each mechanism an advice and consulting relationship or a therapy relationship draws on (Table 2). Select a block or a mark to jump to that mechanism.

Better or Same as Humans

Worse than Human Memory

Not Compared

Semantic Knowledge Confidence Calibration Instrumental Learning Working Memory Priming Retrieval Interference Partner Model Episodic Memory Standing Constraints Temporal Placement Metacognitive Control Schema Formation Classical Conditioning Procedural Memory Reality Monitoring Source Monitoring Schema Application Integrative Inference Prospective Initiation Recognition Feeling of Knowing Consolidation Context-Dependent Retrieval Autobiographical Memory Involuntary Retrieval Reconsolidation Non-Associative Learning Forgetting Motivated Forgetting

Advice and Consulting (Table 2a)

Therapy (Table 2b)

AI Memory Performance

  • Exceeds Human
  • Comparable to Humans
  • Below Human
  • Not Yet Compared
  • Comparison Not Possible
Show as a table
Mechanism AI memory performance Advice and Consulting (elements of 9) Therapy (elements of 15)
Semantic Knowledge Exceeds Human 3 1
Confidence Calibration Comparable to Humans 3 2
Instrumental Learning Comparable to Humans 1 3
Working Memory Comparable to Humans 1 2
Priming Comparable to Humans – 1
Retrieval Interference Comparable to Humans – –
Partner Model Below Human 6 13
Episodic Memory Below Human 4 13
Standing Constraints Below Human 4 6
Temporal Placement Below Human – 8
Metacognitive Control Below Human – 3
Schema Formation Below Human – 2
Classical Conditioning Below Human – –
Procedural Memory Below Human – –
Reality Monitoring Not Yet Compared 2 5
Source Monitoring Not Yet Compared 3 3
Schema Application Not Yet Compared 2 2
Integrative Inference Not Yet Compared 2 2
Prospective Initiation Not Yet Compared – 3
Recognition Not Yet Compared – 2
Feeling of Knowing Not Yet Compared 1 1
Consolidation Not Yet Compared – 1
Context-Dependent Retrieval Not Yet Compared – 1
Autobiographical Memory Comparison Not Possible 3 4
Involuntary Retrieval Comparison Not Possible – 4
Reconsolidation Comparison Not Possible – 1
Non-Associative Learning Comparison Not Possible – –
Forgetting Comparison Not Possible – –
Motivated Forgetting Comparison Not Possible – –

Memory Systems · Immediate

Working Memory

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Comparable to Humans approximate provisional

Memory Systems · Declarative

Semantic Knowledge

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Exceeds Human provisional

Memory Systems · Declarative

Episodic Memory

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Below Human approximate provisional

Memory Systems · Non-Declarative

Priming

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Comparable to Humans approximate

Memory Systems · Non-Declarative

Non-Associative Learning (Habituation and Sensitization)

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Comparison Not Possible

Memory Systems · Non-Declarative

Classical Conditioning (Delay and Trace)

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Below Human provisional

Memory Systems · Non-Declarative

Instrumental Learning (Action–Outcome and Habit)

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Comparable to Humans approximate provisional

Memory Systems · Non-Declarative

Procedural Memory

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Below Human provisional

Representational Structures · Schemas

Schema Formation

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Below Human approximate provisional

Representational Structures · Schemas

Schema Application

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Not Yet Compared

Representational Structures · Person Models

Autobiographical Memory

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Comparison Not Possible

Representational Structures · Person Models

Partner Model

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Below Human approximate

Memory Processes · Acquisition and Transformation

Consolidation

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Not Yet Compared

Memory Processes · Acquisition and Transformation

Reconsolidation

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Comparison Not Possible

Memory Processes · Acquisition and Transformation

Integrative Inference

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Not Yet Compared

Memory Processes · Retrieval and Surfacing

Recognition (Familiarity and Recollection)

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Not Yet Compared

Memory Processes · Retrieval and Surfacing

Temporal Placement

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Below Human approximate provisional

Memory Processes · Retrieval and Surfacing

Context-Dependent Retrieval

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Not Yet Compared

Memory Processes · Retrieval and Surfacing

Involuntary Retrieval

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Comparison Not Possible

Memory Processes · Retrieval and Surfacing

Standing Constraints

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Indirectly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Below Human provisional

Memory Processes · Retrieval and Surfacing

Prospective Initiation

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Not Yet Compared

Memory Processes · Degradation

Forgetting

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Comparison Not Possible

Memory Processes · Degradation

Retrieval Interference

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Comparable to Humans approximate provisional

Memory Processes · Degradation

Motivated Forgetting

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Comparison Not Possible

Metacognition · Attribution

Source Monitoring

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Not Yet Compared

Metacognition · Attribution

Reality Monitoring

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Not Yet Compared

Metacognition · Confidence

Feeling of Knowing

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Indirectly Measured

AI Memory Performance: Not Yet Compared

Metacognition · Confidence

Confidence Calibration

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Comparable to Humans provisional

Metacognition · Control

Metacognitive Control

Full write-up →

Definition

Neural Substrate

Human Measurement

Level of Measurement: Directly Measured

AI Measurement

Level of Measurement: Directly Measured

AI Memory Performance: Below Human approximate provisional