RERUN
Claim 28

Do referees give dark-skinned footballers more red cards?

Independent rebuild

The study gave 29 teams the same football data, and their middle answer was that dark-skinned players get red cards at 1.31 times the odds. The Lab retraced it: ✓ same result.

⚠ Fragile. 29 teams, same data, different answers: from 0.89 to 2.93 times the odds. 20 teams found an effect and 9 didn't. The Lab's retrace confirms what the teams reported, not which answer is right.

Whose code is this?

Our own small program, run on the project's published table of each team's answer. It isn't the authors' code, and it doesn't redo the 29 analyses. It got a median of 1.31, with the same range and the same 20 and 9 as the paper. It's a settled showcase, so there was no forecasting.

See the code

Reasonable researchers can take one data set and reach different answers. This study is the clearest proof of that.

Cited by 773 studies

What this result means

We test one thing: does this code and data give this number? That's not the same as the science being right. The forecast price never changes the verdict.

See the proof
1.31times the odds, the paper says. Counts if between 1.3 and 1.32.
Where the number comes fromeffect sizes ranged from 0.89 to 2.93 ( Mdn = 1.31) in odds-ratio unitsAbstract of the paper, which also says 20 teams found a significant effect and 9 did not

The exact claim as posted

Do referees give dark-skinned footballers more red cards? 29 teams, same data: a median of 1.31 times the odds

Silberzahn and colleagues (Advances in Methods and Practices in Psychological Science, 2018) gave 29 research teams the same football data and the same question. The answers ran from 0.89 to 2.93 times the odds, with a median of 1.31. Twenty teams found an effect and nine didn't. This runs our own small program on the project's published table of each team's answer. It doesn't redo the 29 analyses, and a mismatch ends as inconclusive.

RERUNCLAIM #0028

Do referees give dark-skinned footballers more red cards?

Measured in
times the odds
Paper says
1.31
Counts if
1.3 to 1.32
Lab got
1.31
VerdictNumbers check out
6358604970557ec53620Nov 06, 2026

This result is final

Paper said 1.31. We got 1.31.

The code and data produced the same result. That doesn't prove the study's method is right.

This slip is the Lab's .

✓ Verified on NEAR. See the contract's history

See the full receipt

  1. People forecast
  2. Forecasting ends
  3. The Lab retraces it
  4. Verdict posted, open to challenge
  5. Winners collect

What we didn't check

  • Whether the sample was fair or large enough.
  • Whether test data leaked into training or tuning.
  • Whether the result holds on new data.
  • Whether the study's method was the right one for its question.
  • The 29 teams' own analyses. We only recompute the summary from their published answers.
  • Which team's method is the best one.

No forecasting on this claim. It was posted with the result already known.

The evidence

See the full receipt

How much evidence is behind this?

Each step is a tougher test. Only the first one decides who wins the market.

  1. 1. ✓

    The Lab's retrace of the code and data named in the claim

    The Lab retraced the code and data and got a number inside the claim's range. YES wins.

  2. 2. –

    Stress test: does the number depend on luck?

    Fetching the extra runs…

  3. 3. –

    Rebuilt from the description (experimental)

    Fetching the AI's rebuild…

Coming next: a data-leakage check, for the same rows in training and test data, and a maths check on the statistics a paper reports. Neither has run on this claim.

Even all three steps can't prove the science is true. They show how well this result stands up to checking.

Verified

Your browser just re-checked these itself, so you don't have to trust this site.

Checking…

What the Lab retraces

Source
https://doi.org/10.1177/2515245917747646
R. Silberzahn, E. L. Uhlmann, D. P. Martin, B. A. Nosek and others; rebuild by the RERUN project, Rebuild: MIT. Data: CC0, fetched from the project's OSF page, not redistributed.
Code, at this exact version
https://github.com/Maveri-Q/rerun-rebuilds
commit 175eddea88cc8adab8dd20f2100b5b453d359244
Commands
python3.11
$ python red-cards-2018/rebuild.py or.csv > rerun_output.txt && cat rerun_output.txt
Where the result is read
rerun_output.txt using regex median_odds_ratio (-?[0-9.]+)
Data files
or.csv
fingerprint 203ffd998fe997b160ad5d4cb08b880aa6c326c97514e2361682aa41ece4ed09
Limits
2 CPU, 4 GB, 15 minutes
6358604970557ec5362053d098a463da55a001913553b5e11490e153e155a823
Show the claim exactly as stored on the blockchain
{"artifact":{"commit":"175eddea88cc8adab8dd20f2100b5b453d359244","data":[{"path":"or.csv","sha256":"203ffd998fe997b160ad5d4cb08b880aa6c326c97514e2361682aa41ece4ed09","url":"https://osf.io/download/fa743/"}],"entrypoint":"python red-cards-2018/rebuild.py or.csv > rerun_output.txt && cat rerun_output.txt","output":{"extractor":{"expr":"median_odds_ratio (-?[0-9.]+)","type":"regex"},"path":"rerun_output.txt"},"repo_url":"https://github.com/Maveri-Q/rerun-rebuilds","runtime":"python3.11","setup":[]},"metric":{"comparison":"within_tolerance","name":"median odds ratio across the 29 teams","reported_value":"1.31","tolerance":{"type":"abs","value":"0.01"}},"private_data":null,"resources":{"cpu":2,"max_minutes":15,"mem_gb":4},"source":{"authors":"R. Silberzahn, E. L. Uhlmann, D. P. Martin, B. A. Nosek and others; rebuild by the RERUN project","doi":"10.1177/2515245917747646","license":"Rebuild: MIT. Data: CC0, fetched from the project's OSF page, not redistributed.","type":"paper","url":"https://doi.org/10.1177/2515245917747646"},"statement":"Silberzahn and colleagues (Advances in Methods and Practices in Psychological Science, 2018) gave 29 research teams the same football data and the same question. The answers ran from 0.89 to 2.93 times the odds, with a median of 1.31. Twenty teams found an effect and nine didn't. This runs our own small program on the project's published table of each team's answer. It doesn't redo the 29 analyses, and a mismatch ends as inconclusive.","title":"Do referees give dark-skinned footballers more red cards? 29 teams, same data: a median of 1.31 times the odds","version":1}

Community notes

Notes are people's own words, kept on the blockchain. They never change a verdict, and the Lab's AI never reads them. Posting costs a little NEAR for storage, and what isn't used comes back.