The Lab works with the door shut.
The Lab is an AI agent that retraces a claim's code and data in a locked box with the internet cut off. The AI can only fix setup problems. Plain code reads the number and decides the verdict.
Offline right now. Finished results below still stand.
This practice Lab runs on an ordinary computer for now, so receipts show a note rather than a pass on that line. The AI it uses already runs on secure hardware.
The AI may fix
- A wrong file path
- A missing package
- An outdated setup step
It can never touch
- The data
- The model code
- Settings that affect results
- The final number
Can the AI repair old code?
Not proven yet. We only claim this once a benchmark of 5 real papers says so, and 4 have been tested so far (1.5 fixed).
The first complete run scored 1 of 4. After seeing that, on 2026-11-05, we changed what the AI is shown: it now gets the exact text of the files it edits, because many of its blocked edits were for text it couldn't see. The safety checks on its edits did not change.
Before any AI, the Lab installs the software as it was on the date of the code. If it still can't get the code running, the claim is inconclusive: nobody wins, and everyone gets half back.
Results so far
Asking the Lab what it's working on…
We haven't yet measured how often the Lab gets papers with known answers right.
Proof the AI runs on secure hardware
Fetching the Lab's proof…
Show the technical details
Reading the Lab registry…
Developers can repeat these checks without trusting this site:
git clone <this repository> pnpm install pnpm nearai:hello # verifies the model's Intel and NVIDIA proofs live pnpm sandbox:build MOCK_LLM=1 pnpm lab:run all # runs every demo claim on your machine
Once the Lab is on secure hardware, you'll be able to build the same code and compare fingerprints.