AI Referee Paper Leaderboard (Georgetown University)

Georgetown UniversityBuilt by: research, non-profit or individualStatus: LiveYear: 2026

Overview

The AI Referee Paper Leaderboard, run by the AI, Analytics, and the Future of Work Initiative at Georgetown University, uses an AI referee to score recent finance and economics working papers from SSRN and ranks the top 100. The referee is Claude Opus 4.8, which gives each paper an overall score from 0 to 100, scores it on five dimensions (significance, originality, correctness, data and methodology, and exposition) and writes a full report covering strengths and weaknesses, suggested improvements and its own assessment of its confidence.

The project describes itself as a real-time research experiment to test whether frontier AI models can identify promising research before conference and publication outcomes are known. The team plans to check whether papers with higher initial scores are more likely to be accepted at selective conferences or published in leading journals.

Notes and quotes

  • The project says it is “not an endorsement of replacing peer reviewers, editors, or conference committees with AI”.

Sources

This entry was written with the help of AI, from the sources listed. Please check the sources before relying on it, and let us know if anything needs correcting.