The Agentic Reviewer is a free tool at paperreview.ai from the Stanford ML Group, built by Yixing Jiang and Andrew Ng and released in November 2025, that gives research papers AI-generated review feedback grounded in related arXiv papers. Users upload a PDF of up to 10MB, of which the first 15 pages are analysed, and can choose a target venue. The workflow converts the paper to Markdown, writes search queries, retrieves related arXiv papers and summarises them, then writes a review from a template. For submissions aimed at ICLR it also gives an overall score, which a regression produces from ratings on seven dimensions.
Benefits & outcomes
- The authors report that on 147 held-out ICLR 2025 submissions, the Spearman correlation between the AI score and one human reviewer was 0.42, compared with 0.41 between two human reviewers.
Notes & quotes
- The authors report an AUC for predicting acceptance of 0.84 using one human score and 0.75 using the AI score, and note that this is not a direct comparison because the human scores influenced the actual decisions.
- The tech overview says that grounding in arXiv should work better in fields such as AI, where research is widely posted there, and less well elsewhere.
- The site warns that reviews are AI-generated and may contain errors, and the tool supports English-language papers only.
Publisher / company: Stanford University
Date of case study: 2025
Status: Live
Built by: Research, non-profit or individual
Sources: