Research

Where AI still needs help

Taking the Hendrycks CHC battery as the most precise public score, and reading it against Chollet, Morris, Legg, and METR: the remaining work is not “more knowledge.” It is storage, retrieval discipline, working memory as an object, novel reasoning, and faculties a text lattice cannot fake.

General Knowledge

Kmedium scaffold

Breadth of factual understanding: commonsense, culture, science, social science, history.

The gap: Parametric knowledge is wide but stale, uneven, and unowned by a particular life.
The prosthesis: Semantic memory plus retrieval tools can hold private, up-to-date facts the weights never saw.
The limit: Does not give the model a living world-model; it still quotes rather than inhabits knowledge.

Layers aimed here: Semantic memory, Tool use, World model

Reading & Writing

RWlow scaffold

Consuming and producing written language, from decoding to composition and usage.

The gap: Frontier models already match or exceed a well-educated adult on most literacy tasks.
The prosthesis: Style constitutions and revision loops polish voice; they do not create literacy.
The limit: Long-form consistency still drifts without an external outline.

Layers aimed here: Constitution

Mathematical Ability

Mmedium scaffold

Arithmetic through calculus, probability, and formal manipulation.

The gap: Symbolic slips persist when the model does arithmetic in prose.
The prosthesis: A calculator and program-of-thought tools catch the slips the weights still make.
The limit: Novel proof strategy remains a reasoning problem, not a tool problem.

Layers aimed here: Tool use

On-the-Spot Reasoning

Rhigh scaffold

Flexible attention on novel problems: deduction, induction, theory of mind, planning, adaptation.

The gap: Fluid intelligence on unseen puzzles (ARC-style) and multi-step plans still collapse without search.
The prosthesis: Planner, debate, and reflexion implement System-2 search around the model — the move Claude Code, Deep Agents, and Tree-of-Thoughts all make.
The limit: Search over tokens is not Chollet's skill-acquisition efficiency. The model does not get better at the next novel task.

Layers aimed here: Hierarchical planner, Reflexion critic, World model, Metacognition, Skill library, Internal debate, Goal stack

Working Memory

WMhigh scaffold

Maintain and manipulate information in active attention across modalities.

The gap: Context windows are a poor working memory: they rot, they are not addressable, they are not operated on.
The prosthesis: A scratchpad the agent can read, write, and edit is the standard ACI from SWE-agent onward.
The limit: Cross-modal working memory still needs the model's native vision/audio, not a text board.

Layers aimed here: Working memory

Long-Term Memory Storage

MShigh scaffold

Continually learn new information — associative, meaningful, and verbatim. The paper's sharpest remaining zero.

The gap: GPT-5 still scores 0. Weights do not update from a conversation. Every session is amnesia.
The prosthesis: Episodic and semantic stores (Letta, Mem0, this harness) persist experience outside the weights — the only working prosthetic for MS today.
The limit: Hendrycks et al. call this a capability contortion. RAG is not LoRA. The model does not become the memory; it consults it.

Layers aimed here: Episodic memory, Semantic memory, Skill library

Long-Term Memory Retrieval

MRhigh scaffold

Fluency and precision of access, including the refusal to confabulate.

The gap: Hallucination is a retrieval-precision failure. Search tools hide it without fixing it.
The prosthesis: Ground answers in stored memories; metacognition forces a confidence tag when the store is empty.
The limit: The generator can still ignore the retrieved note. Precision is a discipline, not a guarantee.

Layers aimed here: Episodic memory, Semantic memory, Reflexion critic, Metacognition

Visual Processing

Vlow scaffold

Perceive, analyze, reason about, generate, and scan visual information.

The gap: Scaffolding in text cannot see. Native multimodal models closed some of this; GUIs and spatial scans remain weak.
The prosthesis: Computer-use and screenshot loops (Claude managed agents, Operator-class systems) are the visual prosthesis.
The limit: This web harness does not embody a pointer. Vision stays a model capability, not a lattice one.

Layers aimed here: none — model-native

Auditory Processing

Alow scaffold

Discriminate, recognize, and work creatively with speech, rhythm, and music.

The gap: Speech I/O is now a product feature; rhythmic and musical judgment are not.
The prosthesis: TTS/STT wrap the model. They do not give it an ear.
The limit: Out of scope for a text harness. Honest gap.

Layers aimed here: none — model-native

Speed

Smedium scaffold

Simple cognitive tasks quickly: perceptual speed, reaction, number facility, pointer fluency.

The gap: Token generation is fast at easy tasks and slow at hard ones; it is not human reaction time.
The prosthesis: Tools (calc, clock) and caches make easy operations exact and cheap.
The limit: A deliberative lattice makes hard tasks slower on purpose. Speed is not the goal of proto-AGI.

Layers aimed here: Tool use

What scaffolding cannot do

Weight-level continual learning without catastrophic forgetting. A database is not a LoRA. Hendrycks et al. are explicit about this.
Chollet-style skill-acquisition efficiency. Test-time search can look like fluid intelligence until the next distribution shift (ARC-AGI-2).
Real vision, audition, and pointer fluency. This chamber is text.
Economic AGI in the OpenAI-charter sense. A logged-in workspace is not a firm.
A self that persists because it changed, rather than because we stored a note.