Albert Kao (Harvard University)
Abstract. The ‘wisdom of crowds’ (where combining many independent opinions can result in improved accuracy in estimation tasks) is a common phenomenon that has been observed in species as diverse as fish, ants, and humans. However, there is still a debate, dating back to Sir Francis Galton in 1907, regarding which aggregation measure (such as the arithmetic mean or the median) will most reliably generate an accurate collective estimate. Here, we combine a meta-analysis of multiple independent datasets of the classic jellybean estimation task and computational simulations to look more closely at the assumptions underlying the wisdom of crowds. We demonstrate that systematic estimation biases strongly affect the accuracy of aggregation measures and subsequently devise new measures that counteract the effect of these biases, which are demonstrated to result in improved collective estimates. We then speculate on the general applicability of this methodology across human decision-making contexts.