The Replication Engine

The Replication Engine is an AI system that the Institute for Replication (I4R) is developing to check published research automatically, by reproducing a paper’s computational results, looking for coding errors and testing whether the main findings hold under robustness checks. Abel Brodeur, chair of I4R, and Bruno Barbarioli proposed it in August 2025 in a paper published by the Institute for Progress, and by February 2026 I4R had built a first version and run its first experiments. In June 2026 the team reported a benchmark on 74 economics and political science papers using GPT-5.5 and Claude Opus 4.7, and said a beta version would be available soon.

Benefits & outcomes

  • I4R reports that across the 74 papers the engine agreed with the reference on whether the code ran about 99% of the time.
  • I4R reports that more than 90% of the robustness checks the engine proposed were judged valid and new, with hallucination rates below 2%.

Notes & quotes

  • I4R says detecting mismatches between reported and regenerated results is still difficult and needs expert judgement.
  • The original proposal set out a three-year first phase costing about $10 million, with targets of 2,500 papers in the first year and 15,000 verified papers by the third.
  • The Institute for Progress says a bill based on the Replication Engine is being drafted for introduction in Congress.

Publisher / company: Institute for Replication
Date of case study: 2025
Status: Pilot
Built by: Research, non-profit or individual
Updated: June 2026

Sources:

Contact
Contact