TL;DR: On 8 October 2026 Memento 3 reported 100.0 on all 25 public ARC-AGI-3 games. The language model stayed frozen. A notebook changed. The public ceiling was already occupied in July.
A headline that says an AI cleared every public game is easy to hear as “the model got smarter.” It did not. A model, here, is a huge file of stored numbers plus a rule for guessing the next chunk of text. A parameter is one of those numbers. A weight is the same number, said as something stored in the file. Frozen means those numbers are not edited while the agent plays. The paper’s loop updates files next to the model. The model is the pen. The files are the memory.
What do you have to know before the score means anything?
Three words, in order. A grid. A button. A person playing once.
Before the benchmark (a fixed test with a published scoring rule), there is just a picture. Older ARC tests show you a small coloured grid and a second grid, and ask you to guess the rule that turned the first into the second. That is a static puzzle. Nobody has to act.
ARC-AGI-3, announced by ARC Prize on 25 March 2026, drops the worked example. Greg Kamradt’s launch post says there are no instructions, no rules, and no stated goals. A person, or a program, is placed in a turn-based grid and has to find out how it works by pressing buttons. The same post said humans score 100% and frontier AI (the best general models of that week) scored 0.51%. That gap is why the test exists. It is not a score from this week.
The official games docs cap a frame at 64 by 64 cells. A cell is one square. Each cell is an integer from 0 to 15, so there are 16 colours. The paper treats the picture the agent sees as a 64 by 64 colour grid. Sometimes, as in the last level of the game ls20, the paper says fog hides part of the board, so the current frame is not the whole world.
An action, in this test, is one press sent to the game. The actions docs name a small set. RESET starts over. ACTION1 through ACTION4 are four simple buttons, mapped for humans to up, down, left, and right, but the agent is not told that mapping. ACTION5 is a fifth button whose meaning changes by game. ACTION6 is a click that also needs an x and a y, each from 0 to 63. ACTION7 is undo. Internal work does not count. Writing Python, rereading the log, and thinking are free. Only the press is on the bill.
A level is one stage of a game. Clear it, and the next stage can add a new mechanic (a new rule the old theory does not cover). There are 25 public games and 183 levels in the table the paper prints. Public means these games are the ones anyone can already see. A hidden set, or a private set, would be games the system was not tuned against. This paper does not report one.
What is RHAE, in one worked example?
A score about how many presses you used, compared with a person who had never seen that level.
RHAE (relative human action efficiency, said “ray”) is the number ARC Prize publishes for this test. The human baseline is not a speedrun and not an average. The scoring docs say testers play each level for the first time. The baseline is the upper median: line the people up by fewest presses, and take the upper of the two middle results. Four players, third place is the line. Five players, third place is still the line.
For one level the raw score is (human presses divided by agent presses) squared. Ten and ten is 1.00. Ten human and twenty agent is 0.25. Ten and one hundred is 0.01. Squaring is the punishment. Being a bit slower is fine. Being much slower falls off a cliff. The per-level score is then capped at 1.15, so one clever shortcut cannot carry a sloppy game.
A game score is a weighted average of the level scores. The weight is the level number. Level 1 is light. The last level is heavy. If you skip the last level, you cannot reach 100 no matter how tidy the early levels were. The docs give the arithmetic: finish 4 of 5 levels and the ceiling is (1+2+3+4) / (1+2+3+4+5) = 10/15, about 67%, even if those four were perfect. The total is the average of the game scores. 100 means every game, every level, at least as efficient as that human line.
One more split, because people mix the bills. The action count is presses. The token count (tokens are chunks of text the model reads and writes, and the thing an API prices) is a different bill. Memento 3 reports presses. It does not report tokens or dollars. Kepler, a different system discussed below, reports both.
What did Memento 3 actually run?
A frozen model, a prose file, a Python file, and a replay that must be exact.
The paper is arXiv:2610.11794, submitted 8 October 2026. The authors are Haoyu Zhao, Zhengxu Yu, Zhiyuan He, Meng Fang, Rasul Tutunov, Haitham Bou-Ammar, Weilin Luo, and Jun Wang, at University College London, Huawei Noah’s Ark Lab in the UK, and the University of Liverpool.
They call the method Code as Model. Guess the rules in sentences. Turn the sentences into a program that can predict the next grid. Keep the program only if it can replay the past exactly. Use the program to plan the next presses. When the world disagrees, fix the sentences first.
They ran it inside Claude Code (Anthropic’s coding agent, a program that can edit files and run commands) with Claude Opus 5 as the backbone and reasoning effort set to extra-high. Reasoning effort is a switch for how long the model works before it answers. The harness (the surrounding program that decides when the model is called, what files it may edit, and what gets sent to the game) is built from Sergey Rodionov’s baseline1 codebase. The model’s parameters stay fixed. The paper’s phrase is that the underlying LLM (large language model) remains fixed.
The steps, in the paper’s order:
- Observation. A press returns a new grid. That grid is appended to the history.
- Reflection. The model chooses to keep the current theory, revise the rule, or only repair the code.
- Rule revision. If the rule is wrong, the prose changes first.
- Compilation. Compilation, here, does not mean a C compiler. It means the model writes Python that is supposed to mean the same thing as the prose.
- Verification. Two gates. The model judges that the Python is still faithful to the prose. A replay must reproduce the observed transitions. The paper’s bar is cell-exact: every cell of the 64 by 64 grid, on every later frame, starting from the grid you see after a reset.
Only then may the planner use the code. The planner searches the predicted game for a short path to the goal it has inferred. It sends that path one press at a time. The first prediction that misses stops the plan. The miss is the new evidence. That stop is the part worth copying even if you copy nothing else.
The two files have names. world_model.md is the rulebook: objects, what the buttons seem to do, how things move, what the goal seems to be, and the alternatives still open. world_model_engine.py is the executable theory: given the current idea of the state and a press, predict the next grid and whether the level ended. A rule change is written to the Markdown before the Python is touched. An implementation bug can be fixed in the Python while the prose stays.
Git is the third object, and it is easy to skip. After each iteration the controller commits the workspace and tags the commit with the iteration number. The agent can diff two versions of the rulebook, or restore a Python file a later edit broke. Without that, every rewrite would destroy the theory it replaced. Memory, here, is version control. It is not a longer prompt.
What number did they actually print?
100.0, on the public 25, in 7,518 presses. That is 44% of the human count.
The human total in their table is 17,135 presses. 7,518 divided by 17,135 is 0.44. The paper states both the fraction and the two totals. Per game, their ratio runs from 0.20 on m0r0 to 0.67 on lf52. The median they give is 0.42. Median means the middle game if you sort the ratios. Every one of the 183 levels was cleared. Mean RHAE is 100.0. Mean is the average.
They compare against other paper-backed rows on the ARC Prize scorecards, counted the same way.
| System | Games finished | Mean RHAE | Presses | How to read the presses |
|---|---|---|---|---|
| People, their table | 25 of 25 | the line | 17,135 | Upper-median first tries, summed |
| Continual Harness | 3 | 20.5 | 14,558 | Did not finish. Not a solution cost. |
| DreamTeam | 6 | 38.1 | 9,789 | Did not finish. |
| OPINE-World | 20 | 78.4 | 12,859 | Left 5 games unfinished. |
| NOOA | 19 | 85.1 | 11,697 | Left 6 games unfinished. |
| baseline1 | 25 | 99.0 | 8,347 | Finished. 0.49 of the human count. |
| Memento 3 | 25 | 100.0 | 7,518 | Finished. 0.44 of the human count. |
Read the last two rows as the only fair press comparison. baseline1 already finished every public game at 99.0, using 8,347 presses. Memento 3, starting from that codebase, adds the prose rulebook and the rest of the loop, and prints 100.0 at 7,518. That is about 829 fewer presses than baseline1, about 10% fewer. It is a real improvement on the public set. It is not a new category of result. The category “finished all 25” already had a member.
Against the bare model, the paper says its agent is 59.3 RHAE points above Claude Opus 5 with the ARC Prize Standard harness. The citation is the ARC Prize Opus 5 results page, which the paper says it accessed on 1 October 2026, labeled as the public demo at high reasoning effort. Do not turn 59.3 into a second score by subtraction. Separately, on 24 July 2026, ARC Prize posted that Opus 5 was at 30.2% on the public demo. Those can both be true at different dates, or under a different harness setting. Do not merge them.
Was 100 news on 8 October?
No. The public board had been at the ceiling since July. The paper says so, in its own related work.
On 9 October 2026, the ARC Prize community leaderboard for the ARC-AGI-3 Public Demo listed:
- Tycho, Jens Lehmann, Andrei Aioanei, and Sahar Vahdati, 100.0%, $2,986, dated 29 July 2026, with code.
- Retrodict, Ryan Brown, 99.9%, $654, dated 19 July 2026, with code. Its README says hypotheses must replay the recorded frames before a live press is spent.
- baseline1, Sergey Rodionov, 99.0%, $400, dated 15 July 2026, with code. A coding agent that builds and verifies an executable Python world model, then plans through it.
ARC Prize’s note on the wider board is that semi-private ARC-AGI-1 and ARC-AGI-2 results are run by ARC Prize, and everything else is a public set and self-reported. Treat a community 100 as a public-set claim with a scorecard, not as a hidden exam. Use it to date the ceiling, not to crown a winner.
Kepler is the preprint to put beside Memento 3, because it is specific about money. arXiv:2610.00834, Wensen Wu, submitted 30 September 2026. The abstract says one frozen Claude Opus 5 configuration got a server-verified 100.00 RHAE on all 25 public games, with no per-game model picking and no reruns chosen because the score looked bad. Retained runs used 8,256 environment actions, of which 7,292 were in scored levels. The bill, from local logs, is 858.0 million tokens, 97.37% of them cache reads (a cache read is a repeated prefix the provider does not fully recompute), and $777.72 at 1 September 2026 list rates. Kepler’s files are world_model.py and notes.md. That is the same family of idea. It is not the same program. Do not subtract 7,292 from 7,518 as if the two action definitions were one ruler. Kepler also says the sentence the headline writers skip: public-set score alone has limited discriminative value.
Memento 3’s related-work section is just as plain. It says Tycho and Rodionov’s component study already reported saturation, or near-saturation, of the public set with stronger frontier models. Then, in the paper’s words: public-set completion alone cannot isolate the contribution of a world-model architecture. Comparisons also have to account for which model, how much reasoning budget, action efficiency, whether a rerun is stable, and held-out evaluation. Held-out means games you did not tune on. That paragraph is the paper reviewing its own headline.
There is a third board, and it should not do sneaky work. On three.arcprize.org, the verified table is described as open-source agents on a private set of three unreleased games from the developer-preview competition. On 9 October, StochasticGoose was at 12.58% there, and the rows showed no dates. 12.58% is not the current hidden score of the field, and it is not Memento’s score. The point is that Memento did not publish a number on that kind of table.
What is new, once you throw out the ceiling?
A named split between the sentence and the code. A small ablation. A population on one game. A Pong controller that keeps playing after the model hangs up. And several missing numbers.
An ablation is a rerun with one piece removed, so you can see what that piece was doing. They deleted the prose rulebook and left the code world model, the harness, the replay checker, the planner, the backbone, and the reasoning effort in place. They did this on three games, one from each difficulty band they defined by human presses per level: ft09 (easy, 35), ka59 (medium, 104), and cn04 (hard, 132). They say a full rerun is expensive in time and money, which is why it is three games and not 25.
Both arms still cleared every level and still scored RHAE 100. So the English file is not why the score hit the ceiling. Across the three games, the version with the rulebook used 617 presses rather than 677, a 9% cut, and 678 agent turns rather than 830, an 18% cut. A turn, here, is one loop of the agent, not one press. The prose made the search cheaper. It did not create the ability to finish.
The population test is the other new switch. A population, here, means several rival theories kept at once, not one theory you have married. They share one history. Each planning episode samples one theory. Every theory is checked against the presses that actually happened. A theory that predicted the wrong grid gets repaired. They ran this at N=2, only on wa30, the game with the largest human press count, 1,843. Both the single theory and the pair scored RHAE 100. Presses fell from 899 to 597, which is 0.66 times, and eight of the nine levels were at least as cheap. Level 8 went from 130 to 138, slightly worse. One game, one pair of theories. That is a promising mechanism. It is not a result about all 25 games.
The Pong case study is the sentence that will get clipped wrong. Atari Pong is a two-player paddle game: you move a paddle, a ball bounces, first to 21. The paper says the same rulebook loop can learn a controller (a rule that picks the next move from the current picture, over and over, not a single plan that ends). They evaluate a fixed controller on three openings. It wins 21 to 0 each time. The openings take 3,358, 4,014, and 5,039 actions. Learning cost is 9,504 emulator frames (a frame, here, is one tick of that game). During the exam the language model is not called. The paper also says the model proposes objects and bounce ideas from pretrained knowledge (knowledge already inside the frozen file, from training, before this experiment). Pong is a famous game. ARC-AGI-3’s claim is that the games are novel. Do not carry the 21 to 0 across that gap. A controller that keeps working after the model stops is interesting. It is not “the system invented physics it had never seen.”
What the paper does not include:
- A code or data URL. baseline1, Retrodict, and Tycho do link code from the community board. Kepler calls its harness open source in the abstract.
- A dollar figure or a token count for the 25-game run. The paper says the runs cost substantial time and money. Substantial is not a bill.
- A hidden-set or private-set score. The evaluation section says they evaluate on all 25 public games.
- A claim that the weights changed. The opposite. The LLM remains fixed.
The acceptance rule also has a soft half. Replay is mechanical: the cells match, or they do not. The other half is the model judging that the Python still means the prose. A model judging its own homework is not the same kind of evidence as a cell mismatch. The paper uses both. If you steal one, steal the replay.
What should a working programmer copy?
The gate. Not the score.
If you already run a coding agent on a task with a checkable world, the transfer is ordinary.
- Write the rule in a sentence before you write the function. If they disagree, the sentence is the spec (the description of what the code is supposed to mean).
- Make the guess executable. A function that takes the last state and an action and returns the next state. If you cannot run it, you cannot catch it.
- Replay the whole log before the next real action. Exact means exact. Mostly the same grid is how a wrong plan gets spent.
- Stop at the first miss. Do not finish a plan the world has already contradicted.
- Commit the theory. The paper tags every iteration. A killed idea you can still diff is worth more than a tidy file that forgot it.
- Ask three questions before you repeat a 100. What was the hidden set? What was the bill? Where is the code?
The deeper point, and the one the October headlines blur, is where learning sat. Recursive self-improvement (a loop in which a system uses what it just learned to choose what to learn next) sounds like the model rewriting its own weights. In this paper the model does not. The loop rewrites a hypothesis, checks it against a log, and lets the surviving hypothesis pick the next experiment. That is a real loop. It is also the loop a careful scientist already runs with a notebook. Calling the notebook a new mind is how a 100 from July gets sold again in October.
What I would not do with this
I would not budget a project on the public 100. I would not tell a team the model learned the games. I would not ignore baseline1.
If you need a system that plays novel games, start from a codebase that already finished the public 25 and that you can read. baseline1 is the one this paper says it started from, and the community page links code at a stated $400. Retrodict links code at a stated $654. Tycho links code at a stated $2,986. Kepler publishes a method, a server-verified claim, and a bill. Memento 3 publishes the cleanest account so far of the prose-versus-code split, the git memory, the three-game ablation, and the one-game population. Until there is a repository and a scorecard URL, that account is the product. The agent is not in your hands.
Common questions about Memento 3 and ARC-AGI-3
Did Memento 3 make Claude Opus 5 smarter?
No. The paper says the underlying LLM remains fixed. What changed between moves was a notebook beside the model: a prose rulebook, world_model.md, and a Python world model, world_model_engine.py, both committed to git after each iteration.
What score did Memento 3 get on ARC-AGI-3?
A mean RHAE of 100.0 on all 25 public games, clearing all 183 levels in 7,518 presses. That is 0.44 of the 17,135-press human total in the paper’s table.
Was Memento 3 the first to score 100 on the public ARC-AGI-3 games?
No. On the community leaderboard, Tycho was listed at 100.0% on 29 July 2026, Retrodict at 99.9% on 19 July, and baseline1 at 99.0% on 15 July. Kepler reported a server-verified 100.00 on 30 September.
How much did the Memento 3 run cost?
The paper does not say. It reports presses, not tokens or dollars, and only calls the runs substantial in time and money. Kepler, a similar system, reports 858.0 million tokens and $777.72.
What does the prose rulebook actually add?
Efficiency, not the finish. In the three-game ablation both arms scored 100, but the rulebook version used 617 presses instead of 677 and 678 agent turns instead of 830.