How much psychology research holds up when it's repeated?
✓ Numbers check out
Counts as holding up
The code and data produced the same result. That doesn't prove the study's method is right.
When independent researchers repeated 100 published psychology experiments, they counted how many repeats still showed a statistically significant result and got 36 percent, matching the paper.
The Open Science Collaboration repeated 100 published psychology experiments to see how many would produce significant results a second time. This analysis does not redo any experiments; it recounts the share of repeats that were statistically significant — a result unlikely to be due to chance — using the project's public results table. Of the 97 original experiments with significant results, the paper reports that 36 percent of the repeats were significant too.
An independent rebuild of the counting program, run on the project's public results table, found that about 36 percent of the repeats were statistically significant, matching the paper's figure. This checks that the project's own numbers can be reproduced, not that the 100 experiments themselves hold up. A mismatch here would only mean the recount was inconclusive, and the paper reports other measures of replication that were not checked.
The text was written by a private AI model on NEAR AI (z-ai/glm-5.3-flash). The verdict and numbers come straight from the blockchain, and the AI can't change them.
- What the paper said
- Thirty-six percent of replications had statistically significant results. Abstract of the paper. The authors' own script prints 35/97
- What we got
- 36.0825
- Whose code ran
- Independent rebuild
- How well known
- Cited by 8,484 studies
- What forecasters thought
- Nobody forecast this one. We posted it with the result already known, to show how a famous paper looks.
- Proof
- Read the receipt or see claim 30
What we didn't check
- Whether the sample was fair or large enough.
- Whether test data leaked into training or tuning.
- Whether the result holds on new data.
- Whether the study's method was the right one for its question.
- The 100 experiments themselves. We only recount the project's own results table.
- Whether statistical significance is the right test of a repeat. The paper gives other measures too.
Community notes
Notes are people's own words, kept on the blockchain. They never change a verdict, and the Lab's AI never reads them. Posting costs a little NEAR for storage, and what isn't used comes back.