Abstract: Frontier multimodal language models are rapidly reshaping how we conduct and evaluate science. This talk presents the AAAI-26 AI review pilot, which explored a specific role for AI in the scientific process: peer review. In response to the explosive growth of AI publishing (over 30,000 initial submissions for AAAI-26) and the increasing technical capabilities of state-of-the-art language models, AAAI-26 ran a pilot in which every paper received one clearly labeled AI-generated review. No human reviewers were replaced, and final decisions remained entirely under human control. I will describe what we built: a thorough multi-stage AI reviewing system that integrates multiple tools and techniques, with explicit criteria at each step, along with the infrastructure required to generate AI reviews for the full submission set in under 24 hours. We also conducted an extensive voluntary survey of authors, reviewers, senior program committee members, and area chairs to assess and compare them with human reviews. Overall, respondents found the AI reviews helpful, and on average, they were preferred to human reviews across 6 of 9 criteria, including overall impressions, review focus, technical accuracy, and research suggestions. We also learned about the current limitations of AI in peer review. I will close with lessons learned, opportunities for effective human-AI teaming in peer review, and open challenges in building and evaluating AI assistance for scientific reviewing.

Bio: Joydeep Biswas is an associate professor in the Department of Computer Science at the University of Texas at Austin and Associate Director of Texas Robotics. He leads the Autonomous Mobile Robotics Laboratory (AMRL), where he directs research focused on perception and planning for long-term autonomy in open-world settings. He is a recipient of the NSF CAREER award, the Amazon Research Award, and the JP Morgan Faculty Research Award, and serves as a Trustee of the RoboCup Federation and a Councilor of AAAI. He was an Associate Program Chair for AAAI-26 and led its AI-assisted peer review pilot.