ReviewRobot is a system from Qingyun Wang, Qi Zeng, Lifu Huang, Kevin Knight, Heng Ji and Nazneen Fatema Rajani that assigns a review score and writes comments for review categories such as novelty and comparison with prior work. It builds knowledge graphs from the paper under review, the papers it cites and a broad collection of background papers in the domain, compares the graphs to predict a score and structured evidence for each category, and turns these into natural-language comments using templates derived from human review sentences.
Benefits & outcomes
- The authors report that the score predictor reached 71.4% to 100% accuracy.
- The authors report that domain experts judged 41.7% to 70.5% of the generated comments valid and constructive, and found them better than human-written comments 20% of the time.
Notes & quotes
- The authors suggest that ReviewRobot could assist paper reviewers, program chairs and authors.
Publisher / company: Qingyun Wang, Qi Zeng, Lifu Huang, Kevin Knight, Heng Ji and Nazneen Fatema Rajani
Date of case study: 2020
Status: Not stated
Built by: Research, non-profit or individual
Sources: