The recent AI Publishing Collective event at BCS London, hosted by BCSWomen, and BCS Publishing, brought together experts from across scholarly publishing to discuss how AI is shaping peer review and research integrity.
AI + Preprint = The End of Journals?
Chris Leonard opened the evening with a thought experiment: what would happen if we layered AI-powered peer review onto a preprint server? If automated systems could check for integrity, novelty, and robustness at the level we expect from human reviewers, would journals still be necessary?
Most people’s first reaction was no. Journals still provide cultural and professional value: trusted brands, communities, and the validation that comes from being reviewed by peers. But Chris also pointed out that norms can change. Overlay journals provide peer review and editorial curation on top of existing preprint repositories. Examples like JMIRx (the first PubMed-indexed overlay journal) and DARIAH’s overlay journal using the Episciences platform exist, but they’ve struggled to gain widespread adoption. Perhaps now is the time to try this model again?
🔗 Read more from Chris in his Scalene newsletter: https://scalene.substack.com
Experimenting with Research Exchange
Sam Parker and Rebecca Windless from Wiley shared how they’re using AI in Research Exchange (ReX) to move from reactive screening toward proactive insight. As Sam put it: “People don’t want alarms, they want actionable insights.”
Their approach combines submission data, publication history, and author profiles through three stages. They identify researchers through trust networks and identity verification, illuminate content by assessing quality and spotting problematic patterns while understanding provenance, and then interpret findings by applying peer review knowledge to turn insights into evidence-based decisions.
Rebecca outlined how AI tools are forcing publishers to rethink not just what is written, but who is writing and whether it’s trustworthy. They distinguish between provenance and quality. Provenance builds trust through transparency in authorship and origins, though disclosure practices must evolve to reflect new AI tools. Quality has two components: style (clarity and coherent reasoning that shows whether authors truly understand their work) and substance (sound methods, rigorous data, and originality that prove research is accurate and ethical).
Research output saw a 212% increase in citable documents from 2004-2024, while reviewer numbers haven’t kept pace. ReX addresses this capacity challenge by supporting reviewer training for new researchers, expanding reviewer pools through better discovery, enabling peer review report transfer between journals, and using AI to guide new reviewers through the review process. [Slides]
Filtered, Not Raw: AI and Transparency
Leslie McIntosh closed the session with a forensic perspective on AI in peer review, drawing on her work in forensic scientometrics – the emerging field dedicated to detecting and preventing research misconduct. She framed her talk around a central question: “How does the publishing ecosystem and peer review in particular influence scientific trust?”
Leslie argued that peer review serves as science’s fundamental trust mechanism, but this system faces new challenges from AI tools that can be used for both defense and exploitation of scientific integrity. Her Thames metaphor illustrated the core challenge: “The Thames is open water, but would you drink it?” Openness alone doesn’t guarantee quality – we need effective filtering systems to ensure what we consume is reliable.
Leslie emphasized the need for “scholarship authentication” using three elements: understanding authorship, funding, and institutional affiliations (provenance); evaluating methodological rigor and data integrity (properties); and expert assessment when doubts arise about authenticity (professionals).
She highlighted concerning patterns in the research landscape, including suspicious author citation networks and the rise of paper mills and predatory publishing that “pollute the information river.” These threaten not just individual papers but the entire foundation of scientific trust.
Her core message: AI in publishing must be transparent and auditable, with decisions that can be explained and verified. She called for collaborative standards across publishers to ensure AI tools defend rather than exploit scientific integrity.
📎 Read more from Leslie in her Forensic Scientometrics newsletter: https://fosci.substack.com
Thanks again to BCSWomen, and BCS Publishing for hosting the discussion that combined practical experience with forward-looking debate about where AI will take peer review next.
Offers from BCS Publishing:
📚 BCS Library Offer: Access the full BCS book collection for £50 a year with code ITNOW: https://shop.bcs.org/page/bcs-subscription-library/
🤖 BCS Research: Public expectations for AI professionals: https://www.bcs.org/articles-opinion-and-research/ai-ethics-and-professional-registrations-in-the-uk-report/
📘 New Book: Getting Started with Tech Ethics: https://shop.bcs.org/page/detail/getting-started-with-tech-ethics/
