Ranking is fundamental to many areas, such as search engine and recommender system optimization, as well as peer grading.

Find out what was taken out of Section 3 of our Algorithmic Contract Design.

Find out what was missing from section 2 of our Algorithmic Contract Design Research paper.

This paper is available on arxiv under CC 4.0 license. It is available under the terms of the Creative Commons license.

We studied the multifaceted problem of designing a crowdsourcing mechanism that efficiently and accurately ranks a set of items using pairwise comparisons

In the following section, we present experiments that test our models and algorithm. We evaluate the impact of our algorithm on the principal’s utility

We now significantly expand our model to study heterogeneous agents with differing, unknown disutilities.

Algorithm 4 takes as input “comparisons”, which is a list of s sublists, each containing the returned results of the binary comparisons performed by an agent.