This paper is available on arxiv under CC 4.0 license. It is available under the terms of the Creative Commons license.

We studied the multifaceted problem of designing a crowdsourcing mechanism that efficiently and accurately ranks a set of items using pairwise comparisons

In the following section, we present experiments that test our models and algorithm. We evaluate the impact of our algorithm on the principal’s utility

Algorithm 4 takes as input “comparisons”, which is a list of s sublists, each containing the returned results of the binary comparisons performed by an agent.