Start here
Findings
What the data shows, one observation at a time. Each finding is stated without judgment, drawn from values recorded as the developers’ documents print them, and linked to every value behind it.
Data as of 26 Sep 2026 · Dataset v0.1 · Methodology v0.1 · Changelog
Summary
One finding is written so far, the latest on 28 Sep 2026. Four more are planned for Phase 3, on restatements, revisions, series and coverage. Each finding states one observation, shows the values behind it in a chart and a table, and says which rows it uses, why, and how they were checked.
Every finding, newest first
33 reported numbers on evaluation awareness come from six kinds of test
How often a model shows signs of recognising a test is measured in six different ways, and the same model gets different numbers from different kinds of test.
Not written yet: planned for Phase 3
- Finding 2Later documents report earlier models again, sometimes with different values (not written yet)
- Finding 3Documents change after release, with and without a changelog (not written yet)
- Finding 4Within one developer and one test, values form a series until the test changes (not written yet)
- Finding 5Who reports what (not written yet)