Every year, tens of thousands of AI research papers compete for a spot at the world’s top academic conferences and a small group of reviewers has to decide which ones make the cut. The system is under serious strain: reviewer pools are stretched thin, scores are inconsistent, and there is growing concern that some of the most important work isn’t getting the recognition it deserves. New research by Weijie Su and Bingxin Zhao, professors of statistics and data science at the Wharton School, and colleagues Buxin Su, Natalie Collina, Garrett Wen, Didong Li, Jianqing Fan, and Kyunghyun Cho points to a simple and overlooked solution: ask the authors themselves to rank their submissions.
Key Takeaways
- Researchers tend to know which of their own papers is most important. In a large-scale experiment at the International Conference on Machine Learning (ICML) 2023, papers that authors ranked highest among their own submissions received twice as many citations over the following 16 months as those they ranked lowest. This held true whether the papers were accepted or rejected by the conference.
- Authors see something reviewers don’t — and the citations prove it. Author rankings were a stronger predictor of future citations than reviewer scores, both before and after the rebuttal process. Reviewer scores reflect technical quality, but tend to reward incremental improvements over work with broader, longer-term significance.
- The most-cited papers in the study were almost always the ones authors ranked first. Among the 22 submissions that went on to accumulate more than 150 citations, roughly the top 1% of all papers in the study, 77% had been ranked first by at least one of their authors. Author rankings were a more reliable identifier of truly influential work than reviewer scores or acceptance decisions.
- Asking authors to rank, not rate, makes the signal harder to manipulate. Authors order their submissions relative to one another rather than assigning each paper a standalone score. That means they can’t simply claim all their work is exceptional. The design encourages honest responses by making it structurally impossible to inflate everything at once.
Real World Application
Short on time? Here’s the takeaway:
Asking authors to rank their own submissions is a simple, low-cost step that identifies high-impact research more reliably than peer review scores alone.
In the ICML 2023 experiment, 1,342 researchers ranked 2,592 submissions by perceived quality right after the submission deadline. Citations were then tracked for 16 months. Papers that authors ranked highest received twice as many citations as those they ranked lowest, exceeding the citation ratio observed between the highest-scored and lowest-scored groups of papers. Moreover, the most-cited paper in the entire dataset, with 979 citations, had been ranked first by its own authors, yet received a middling reviewer score and was accepted only as a poster. The gap makes sense when you consider what reviewers are working with: tight deadlines, high volumes, and an incentive to focus on what’s measurable. Authors, by contrast, have a fuller picture of why their work matters and where it could lead. The results held up after accounting for timing of arXiv posts, self-citations, and conference promotion. Based on these findings, ICML 2026 has incorporated author self-rankings into its formal review process for the first time.
This content was created with the assistance of generative AI. All AI-generated materials are reviewed and edited by the Wharton AI & Analytics Initiative to ensure accuracy, clarity, and alignment with our standards.
About Wharton AI & Analytics Insights
Wharton AI & Analytics Insights is a thought leadership series from the Wharton AI & Analytics Initiative. Featuring short-form videos and curated digital content, the series highlights cutting-edge faculty research and real-world business applications in artificial intelligence and analytics. Designed for corporate partners, alumni, and industry professionals, the series brings Wharton expertise to the forefront of today’s most dynamic technologies.