Reproducibility

Overview

Reproducibility is the ability of an independent researcher to obtain the same results as an original study when using the same data and the same analysis. It is closely related to, but distinct from, replicability, which uses new data to test whether a finding holds.

Why it matters

Reproducible work lets others audit claims, build on them with confidence, and catch errors early. When studies cannot be reproduced, time and funding are wasted chasing results that may not be real.

Key ingredients

• Open data — the inputs needed to re-run an analysis.
• Open code — scripts and notebooks that transform data into results.
• Documented methods — enough detail to follow each step.
• Versioning — a record of how the work changed over time.

Common obstacles

1. Unshared or proprietary datasets.
2. Undocumented "researcher degrees of freedom" in analysis.
3. Software environments that are hard to recreate.

Reproducibility is a property of the evidence, not just the conclusion.

See also

Replication, open access, and peer review.