Schug & Lake (2026)

Are they human? Detecting large language models by probing human memory constraints

arXiv

Find It

DOI

Abstract

The validity of online behavioral research relies on study participants being human rather than machine. In the past, it was possible to detect machines by posing simple challenges that were easily solved by humans but not by machines. General-purpose agents based on large language models (LLMs) can now solve many of these challenges, threatening the validity of online behavioral research. Here we explore the idea of detecting humanness by using tasks that machines can solve too well to be human. Specifically, we probe for the existence of an established human cognitive constraint: limited working memory capacity. We show that cognitive modeling on a standard serial recall task can be used to distinguish online participants from LLMs even when the latter are specifically instructed to mimic human working memory constraints. Our results demonstrate that it is viable to use well-established cognitive phenomena to distinguish LLMs from humans.

Reference

Schug, S., & Lake, B. M. (2026). Are they human? Detecting large language models by probing human memory constraints. arXiv preprint arXiv:2604.00016. https://doi.org/10.48550/arxiv.2604.00016

What Rests on This

1 mechanism cites this source. Each finding below is quoted as it appears on that mechanism's own page.

Working Memory Memory Systems

Measured in AI

Today's strongest systems score far above people only on uneven tests: in a 2026 preprint, 100 online participants had median accuracy of 0.6 on letter serial recall while ten frontier models (e.g., GPT-5.2, Claude Opus 4.1, Gemini 3 Pro) scored near 1.00 with no serial-position or list-length effects, and a 2024 preprint put models at or above the 99.5th percentile on the WAIS-IV Working Memory Index, but in both the letters or digits stayed visible in the prompt