Preprints.ai screens bioRxiv and medRxiv preprints with an automated pipeline and reports three separate signals for each one: Evidence, graded A to E, on whether the methods and results support the claim; Trust, graded A to E, on transparency and reporting markers such as data availability, statistical rigour and citation health; and Novelty, scored 1 to 10, with lower scores for papers more distinct from the literature. Deterministic checks run first, including paper-mill detection, statcheck-style p-value verification, GRIM tests, data availability and retraction cross-referencing. In a full run, eleven model reviewer roles then assess the preprint: a methodologist, a statistician, scientific validity, ethics and transparency, five domain specialists, a devil’s advocate and a positive-evidence reviewer. Scoring follows a rubric, and borderline cases are arbitrated. The bulk screening used across the catalogue runs the deterministic checks and a calibrated judge only.
Notes & quotes
- The site says the reviewer roles share training and prompts, so their agreement is correlated.
- The methodology page says that fabricated but plausible data can score higher than it should, and that physics, mathematics and computer science receive weaker signals than the life sciences.
- The methodology page lists a blinded, field-stratified comparison with human reviewers as still required.
Publisher / company: Preprints.ai
Date of case study: Not stated
Status: Live
Built by: Company
Sources: