Guiding / Research

Intelligence Measurement

Most AI benchmarks judge intelligence by looking at the final answer: did the model solve the problem, pass the test, complete the task, or produce the expected output. That matters, but it is not enough. This is a research direction that asks a deeper question: what kind of internal process produced that answer? It started while studying Google DeepMind's March 2026 AGI measurement framework and hackathon, which break intelligence into cognitive abilities like learning, attention, reasoning, metacognition, executive function, and social cognition. The angle being explored here is that those abilities should not only be measured at the final output layer, but also inside the model's per token generation dynamics, the shape of the model's thinking may matter as much as the answer it gives. This is not a finished AGI test. It is a step toward measuring the internal structure of intelligence.

Guiding / Research

Measuring AI cognition before the final answer appears

Why this mattered

A benchmark that only observes the final answer is still observing the system from the outside. It is like trying to understand the human brain only by asking people questions. Question and answer tests tell us something, long horizon task completion tells us something, but our real understanding of human cognition advanced because we studied the brain itself: regions, pathways, signals, failures, feedback loops, and internal structure. AI needs an equivalent form of internal study. Two models can produce the same answer for very different reasons, one may reason through the structure of the problem, another may match a memorized pattern, another may stumble into the right result by probability, another may generate a plausible answer while its internal state has already drifted away from the actual problem. From the outside these can look similar; from the inside they may be completely different. On top of that, the Stanford AI Index has called out that static benchmarks are being saturated quickly and face reliability and gaming concerns. The question driving this work is whether intelligence can be measured by studying the topology of internal token by token transitions, not only by grading the final response.

Current status

Active research direction. The page separates a testable hypothesis from validated findings and does not claim that internal model states currently provide a settled measure of intelligence.

What Gordon explored

The core thesis: modern AI systems generate responses one token at a time, and each token is the visible endpoint of an internal state transition. At every step the model is moving through a high dimensional cognitive space, selecting, compressing, expanding, redirecting, and stabilizing information before producing the next token. A strong answer produced by a fragile internal process is not the same as a strong answer produced by stable reasoning; a wrong answer that nearly followed a valid path is not the same as a wrong answer produced by total internal collapse; a hallucination may begin long before the false sentence appears on screen. The goal is to detect those differences before the final output is complete.

Origin in the DeepMind AGI framework. The project began while studying Google DeepMind's work around new benchmarks for measuring progress toward AGI, a cognitive abilities framework covering learning, attention, reasoning, metacognition, executive function, and social understanding. That framing is valuable because it moves beyond one dimensional benchmark scores and tries to break intelligence into measurable cognitive faculties. The deeper issue this work is responding to: even cognitive abilities benchmarks are still scored from the outside, on the final answer.

Per token state capture as the basic measurement unit. As the model generates, the system records internal state changes at each token step. The important unit is not only the final response, it is the transition: state → token → next state. This makes it possible to study the model while it is forming the answer, rather than only judging it afterward.

Latent trajectory mapping. The internal states are mapped into a geometric and topological representation so the system can inspect how the model moves through its own hidden cognitive space. The question becomes: what shape did the reasoning process take? Was it stable, fragmented, over compressed, wandering, anchored, recovering, collapsing? This turns intelligence measurement into a structural problem where the final answer is one piece of evidence and the internal path is another.

Structural signal detection. The system looks for structural signatures that may correspond to reasoning quality or reasoning failure, sudden loss of internal separation, abnormal compression of distinct concepts, contradiction like convergence, unstable jumps between semantic regions, brittle confidence formation, prompt context detachment, and failure patterns that appear before the model's text visibly breaks. The purpose is not to replace accuracy benchmarks; it is to add a deeper diagnostic layer underneath them.

Pre output intelligence diagnostics. A mature version of this system would expose diagnostics such as reasoning stability, context retention, semantic coherence, trajectory integrity, failure onset timing, internal uncertainty handling, long horizon drift, and answer confidence versus process quality. That would allow AI systems to be compared not only by what they answer, but by how they internally arrive there, and would create room for adaptive intervention, where a system could pause, reroute, ask for clarification, reduce confidence, or switch strategies before producing a bad answer.

Tone discipline. The work avoids the claim 'this measures true intelligence' because that sounds too absolute for what is actually being explored. The framing used here is more honest: this is a step toward measuring the internal structure of intelligence, instrumentation for machine cognition, not a finished AGI test.

Proof at a glance

A working public facing thesis: AI evaluation should move from answer grading toward cognitive diagnostics, not just what the model said, but what happened inside the model before it said it. That thesis is taking shape as a research direction, not as a shipped product.
An early evaluation architecture direction, prompt or task input as stimulus, per token state capture, latent trajectory mapping, structural signal detection, and pre output intelligence diagnostics, that can be developed in slices instead of having to be built all at once.
A working vocabulary for the work: process quality, early failure detection, reasoning integrity, cognitive load, generalization, benchmark trustworthiness, topology of internal transitions, and pre output diagnostics. Having that language makes it possible to reason about the system, share it with collaborators, and tell which pieces are architecture, which are prototype direction, and which are still research horizon.
Honest open questions remain, what to instrument, how to capture internal state across different model families, how to make trajectory measurements robust and not just suggestive, what counts as a clean structural signature versus noise, and how this should sit alongside cognitive abilities benchmarks rather than competing with them. The goal is not to claim this measures true intelligence. It is to keep building toward measurement that can look at the internal structure of generation and meaningfully compare what is happening underneath the answers.

Project record reviewed August 30, 2026. Where no public source is listed, the page identifies first person or internal evidence explicitly instead of implying independent verification.