Impartial peer review

David Kurokawa, Omer Lev, Jamie Morgenstern, Ariel D. Procaccia

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Motivated by a radically new peer review system that the National Science Foundation recently experimented with, we study peer review systems in which proposals are reviewed by PIs who have submitted proposals themselves. An (m, k)-selection mechanism asks each PI to review m proposals, and uses these reviews to select (at most) k proposals. We are interested in impartial mechanisms, which guarantee that the ratings given by a PI to others' proposals do not affect the likelihood of the PI's own proposal being selected. We design an impartial mechanism that selects a k-subset of proposals that is nearly as highly rated as the one selected by the non-impartial (abstract version of) the NSF pilot mechanism, even when the latter mechanism has the "unfair" advantage of eliciting honest reviews.

Original languageAmerican English
Title of host publicationIJCAI 2015 - Proceedings of the 24th International Joint Conference on Artificial Intelligence
EditorsMichael Wooldridge, Qiang Yang
Pages582-588
Number of pages7
ISBN (Electronic)9781577357384
StatePublished - 1 Jan 2015
Externally publishedYes
Event24th International Joint Conference on Artificial Intelligence, IJCAI 2015 - Buenos Aires, Argentina
Duration: 25 Jul 201531 Jul 2015

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
Volume2015-January

Conference

Conference24th International Joint Conference on Artificial Intelligence, IJCAI 2015
Country/TerritoryArgentina
CityBuenos Aires
Period25/07/1531/07/15

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence

Cite this