The scout finds. A person decides.
Twice a day an AI goes looking for research that shares its code and data. Nothing goes live until a person says yes.
- ReadsPapers with open code, from GitHub, ReScience C and Hugging Face
- ChecksTwo AIs must pick the same number from the paper
- WaitsA person approves it before anyone can forecast
Most papers can't be retraced
Each slip is a paper. A no isn't a judgement on the research, only that the Lab can't test it fairly.
Ready for review
Two AIs found the same number.
No licence
We can't publish without one.
Needs a graphics card
The Lab has plain processors.
Wrong language
Only Python and R run here.
No code to run
No code link in the paper.
No data included
Nothing to run the code on.
Too big
Too slow for a short run.
Still being checked
Another look next scan.
Popularity only sets the order a person reviews in. It can't touch a verdict.
Waiting for review
- Hugging Face PapersPython
Can tiny AI tools figure out why search results disagree as well as giant AI models, but much cheaper and faster?
5/10 ai, 4 upvotesEveryone gets confusing or contradictory search results, and cheaper conflict detection could make AI answers quicker and less costly.
- Number to check
- 67.0
- Licence
- Apache-2.0
- Run command
- uv pip install -e '.[dev,plot]' && bash scripts/download_data.sh && .venv/bin/dmconflict evaluate --split test
The full claim and the scout's notes
On the CONFLICTS benchmark (406 test queries), 11 small 'decision models' typed why retrieved sources disagree. Asked directly with one choice question, they reached 50.0–67.0% accuracy; 10 of 11 matched or beat Gemini 2.5 Flash (61.3%) at far lower cost and latency, while Gemini 3.8 Flash led at 70.0%. Routing each small model's least-confident quarter of queries to Gemini 3.8 Flash tied it for 8 of 11 models at 25–37% of the LLM's cost, and downstream answers were as good as with LLM-typed conflicts. Suggested by the scout agent.
AI note: Eval code re-scores stored JSONL outputs already in the repo, so no API key is needed and reruns are fully deterministic. Requires uv/pip, network for scripts/download_data.sh (CONFLICTS dataset, checksum-verified, Apache-2.0, © Google). Statement values are paper-reported ranges; extractor targets the Liquid d1 Ultra row in comparison_test.md.
- Open licence: Apache-2.0.
- Mostly Python (76% of the code).
- No sign it needs a graphics card.
- Small enough to re-run.
- The number 67.0 appears in the paper's text, read from https://arxiv.org/pdf/2610.36915.
Only the site's owner can approve a suggestion.
Every paper turned down, and why (58)
- [Re] Computational Modelling of Glucose Uptake by SGLT1 and Apical GLUT2 in the Enterocyte
It doesn't link to a GitHub repository, which the Lab needs.
- [Re] A Reservoir Computing Model of Reward-Modulated Motor Learning and Automaticity
GitHub can't identify an open licence.
- [Re] When Does Label Smoothing Help?
Depends on pytorch, which usually needs a graphics card. The Lab only has CPUs.
- [$\\sim$Re] Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress
The repository is 470 MB, too big to re-run quickly.
- [Re] An anatomically constrained neural network model of fear conditioning
It doesn't link to a GitHub repository, which the Lab needs.
- [Re] A Multi-Functional Synthetic Gene Network
Mostly MATLAB. The Lab only runs Python and R.
- [Re] Reproductive pair correlations and the clustering of organisms
Mostly C++. The Lab only runs Python and R.
- [Re] Effective Program Debloating via Reinforcement Learning
It doesn't link to a GitHub repository, which the Lab needs.
- [Re] Spatial constraints
GitHub can't identify an open licence.
- [Re] Three-dimensional wake topology and propulsive performance of low-aspect-ratio pitching-rolling plates
Open licence: BSD-3-Clause. Mostly Python (94% of the code). No sign it needs a graphics card. Small enough to re-run. The AI couldn't find one clear number to check, and how to produce it.
- [Re] On the Relationship between Self-Attention and Convolutional Layers
GitHub can't identify an open licence.
- [Re] Speedup Graph Processing by Graph Ordering
Mostly C++. The Lab only runs Python and R.
- Words Caught in the Heart: Simulating Person-Specific Expression for Self-Opaque Clients in Psychotherapy
GitHub can't identify an open licence.
- IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?
Open licence: MIT. Mostly Python (100% of the code). No sign it needs a graphics card. Small enough to re-run. The AI couldn't find one clear number to check, and how to produce it.
- What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?
Open licence: MIT. Mostly Python (100% of the code). No sign it needs a graphics card. Small enough to re-run. The AI couldn't find one clear number to check, and how to produce it.
- BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion
Depends on cuda, which usually needs a graphics card. The Lab only has CPUs.
- Advancing SVD-based LLM Compression via Layer-Wise Error Model Search
GitHub can't identify an open licence. Depends on torch, which usually needs a graphics card. The Lab only has CPUs.
- DR Pixel: Page-Native Deep Research with Vision-Language Models
Open licence: Apache-2.0. Mostly Python (100% of the code). No sign it needs a graphics card. Small enough to re-run. The AI couldn't find one clear number to check, and how to produce it.
- SIMIT: Self-Improving Vision-Language Models via Imagination at Test-Time
Depends on torch, which usually needs a graphics card. The Lab only has CPUs.
- Where Do Vision Language Models Fail at 3D Reasoning?
Depends on torch, which usually needs a graphics card. The Lab only has CPUs.
- VeriSpoof: Beyond Real or Fake with One Audio LLM
Open licence: CC-BY, read from the licence file. Check this one yourself. Mostly Python (100% of the code). No sign it needs a graphics card. Small enough to re-run. The AI couldn't find one clear number to check, and how to produce it.
- Kierston & KierstonBench: Simulation of Allentown, PA and Evaluation of Language Models in Crime Investigation and Surveillance Tasks
Mostly TypeScript. The Lab only runs Python and R.
- Enhancing Stock Market Information Retrieval with Retrieval-Augmented Generation
GitHub can't identify an open licence. Depends on tensorflow, which usually needs a graphics card. The Lab only has CPUs.
- [Re] Drivers of evapotranspiration from boreal wildfires
Open licence: MIT. Mostly Python (93% of the code). No sign it needs a graphics card. Small enough to re-run. The AI couldn't find one clear number to check, and how to produce it.
- [Rp] Estimating trends and seasonality in coronary heart disease
Mostly SAS. The Lab only runs Python and R.
- [Re] Chaos in a Three-Species Food Chain
Mostly Julia. The Lab only runs Python and R.
- Turns Matter: Advancing Streaming Video RL via Turn-Level Credit
GitHub can't identify an open licence. Depends on cuda, which usually needs a graphics card. The Lab only has CPUs.
- michaelaoash/HAP-RR-git
Turned down by a person: the AI picked 2.2, but the paper's figure is -0.1, and its command and output file were wrong. The checked claim is posted separately. Open licence: CC0-1.0. Mostly R (100% of the code). No sign it needs a graphics card. Small enough to re-run. The number 2.2 appears in the paper's text, read from https://doi.org/10.3386/w15639.
- aloisgeyer/female-hurricanes
GitHub can't identify an open licence.
- OndrejMarvan/chocolate-nobel-eda
GitHub can't identify an open licence.
- laibabasraa-ux/card-krueger-1994-replication
No small public dataset in the repository, so the Lab would have nothing to run on.
- Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems
Depends on torch, which usually needs a graphics card. The Lab only has CPUs.
- Are Frontier VLM Agents Ready to Be Robot Generalists? An Empirical Study with the Embodied Agent Arena
Depends on pytorch, which usually needs a graphics card. The Lab only has CPUs. The repository is 820 MB, too big to re-run quickly.
- MoviEdit: Long-form Multi-Shot Video Editing Made Easy
GitHub can't identify an open licence. Depends on cuda, which usually needs a graphics card. The Lab only has CPUs.
- dylouie/lancet-meta-analysis-verification
Open licence: GPL-3.0. Mostly R (100% of the code). No sign it needs a graphics card. Small enough to re-run. The AI picked 0.35, but the paper doesn't state that number near "pooled RR/aOR estimates".
- dogunbamowo/payne2008_rescue
Open licence: MIT. Mostly Python (100% of the code). No sign it needs a graphics card. Small enough to re-run. The AI couldn't find one clear number to check, and how to produce it.
- Identically Trained Language Models Break on Different Jailbreaks
Open licence: Apache-2.0. Mostly Python (100% of the code). No sign it needs a graphics card. Small enough to rerun. The AI couldn't find one clear number to check, and how to produce it.
- Factorized MLLM Embeddings for Composed Image Retrieval
Mostly none. The Lab only runs Python and R.
- Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training
Open licence: Apache-2.0. Mostly Python (100% of the code). No sign it needs a graphics card. Small enough to rerun. The AI picked 67.56, but the paper doesn't state that number near "Pass@3 under H0 on AppWorld".
- Obstructive sleep apnea event detection using explainable deep learning models for a portable monitor
No small public dataset in the repository, so the Lab would have nothing to run on.
- PCBRouteAgent: How Close Are LLM Agents to Becoming PCB Routing Engineers?
No small public dataset in the repository, so the Lab would have nothing to run on.
- Authorization Drift in LLM Agents: Evaluating Boundary Compliance in Multi-Step Tool Use
Depends on torch, which usually needs a graphics card. The Lab only has CPUs.
- Sketchpad: certified exact arithmetic with a small vision-language model
Mostly TypeScript. The Lab only runs Python and R.
- InstantHMR : Real-Time Human Mesh Recovery on Your Phone
Depends on torch, which usually needs a graphics card. The Lab only has CPUs.
- Sage-Taiyi: A Metacognitive Architecture for Scientific Discovery
GitHub can't identify an open licence. Mostly none. The Lab only runs Python and R.
- WorldFoundry: Towards Unified Infrastructure for World Intelligence
Depends on pytorch, which usually needs a graphics card. The Lab only has CPUs. The repository is 288 MB, too big to rerun quickly.
- P^2-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors
GitHub can't identify an open licence. Mostly none. The Lab only runs Python and R.
- SplatAnything: Gaussian Spacing Normalization for Feed-forward 3DGS across Arbitrary Scenes
Depends on PyTorch, which usually needs a graphics card. The Lab only has CPUs. The repository is 434 MB, too big to rerun quickly.
- Tree-Ising: Repairing Language Models with Bit-Level Ising Edits
GitHub can't identify an open licence. Mostly none. The Lab only runs Python and R.
- The Surprising Effectiveness of Deleting Weights in LLM Post-Training
Depends on CUDA, which usually needs a graphics card. The Lab only has CPUs.
- Market-Conditioned JEPA: Learning from Unpredictable Market Shocks
Depends on CUDA, which usually needs a graphics card. The Lab only has CPUs.
- Sable: A Small-Field Sumcheck Argument for Bit-Exact Inference of Large Language Models
Depends on CUDA, which usually needs a graphics card. The Lab only has CPUs.
- Toward the Era of Agentic Embodied Intelligence
Depends on torch, which usually needs a graphics card. The Lab only has CPUs.
- Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Agentic Visual Reasoning
Depends on CUDA, which usually needs a graphics card. The Lab only has CPUs.
- Evolving Infra: Agent-Adaptive LLM Serving at Scale
Depends on pytorch, which usually needs a graphics card. The Lab only has CPUs.
- WMIP: A Working-Memory-Inspired Programmable Probe of Cumulative State Tracking in Large Language Models
GitHub can't identify an open licence.
- PhysLoop: Harnessing LLMs' Physics Awareness for Self-Improving Video Generation Models
GitHub can't identify an open licence. Mostly none. The Lab only runs Python and R.
- From Large Language Models to Efficient System One Models via Adaptive Computation
Depends on torch, which usually needs a graphics card. The Lab only has CPUs.