Peer-design agents for reliably evaluating distribution of outcomes in environments involving people

In many domains, an autonomous agent needs to reliably predict the distribution of behaviors of a population rather than the behavior of a single agent. For example, when playing the ultimatum game against several unknown opponents from a large known population, the agent can perform better by extracting its best-response strategy based on the distribution of the acceptance value in that population. In this paper, we demonstrate the efficacy of Peer-Designed-Agents (PDAs) for producing a distribution of behaviors that highly resembles the distribution of actual behaviors of a specific population of interest. This is obtained through extensive experiments with more than 700 different individuals and 132 PDAs, using eight game variants from three different domains and two different statistical tests. The analysis of the results demonstrates that PDAs' technology is an effective means for generating a reliable distribution of behaviors of a population of interest, as long as the similarity between the group of PDAs' developers and the latter population is sufficiently high. Moreover, a comprehensive comparison with the results of Elicited-Strategy-Agents (ESAs) shows that there is much more to PDA technology than simply an expression of strategy.