Condorcet's jury theorem

Condorcet's jury theorem is a political science theorem about the relative probability of a given group of individuals arriving at a correct decision. The theorem was first expressed by the Marquis de Condorcet in his 1785 work Essay on the Application of Analysis to the Probability of Majority Decisions.

The assumptions of the simplest version of the theorem are that a group wishes to reach a decision by majority vote. One of the two outcomes of the vote is correct, and each voter has an independent probability p of voting for the correct decision. The theorem asks how many voters we should include in the group. The result depends on whether p is greater than or less than 1/2:

• If p is greater than 1/2 (each voter is more likely to vote correctly), then adding more voters increases the probability that the majority decision is correct. In the limit, the probability that the majority votes correctly approaches 1 as the number of voters increases.
• On the other hand, if p is less than 1/2 (each voter is more likely to vote incorrectly), then adding more voters makes things worse: the optimal jury consists of a single voter.

Proof

To avoid the need for a tie-breaking rule, we assume n is odd. Essentially the same argument works for even n if ties are broken by fair coin-flips.

Now suppose we start with n voters, and let m of these voters vote correctly.

Consider what happens when we add two more voters (to keep the total number odd). The majority vote changes in only two cases:

• m was one vote too small to get a majority of the n votes, but both new voters voted correctly.
• m was just equal to a majority of the n votes, but both new voters voted incorrectly.

The rest of the time, either the new votes cancel out, only increase the gap, or don't make enough of a difference. So we only care what happens when a single vote (among the first n) separates a correct from an incorrect majority.

Restricting our attention to this case, we can imagine that the first n-1 votes cancel out and that the deciding vote is cast by the n-th voter. In this case the probability of getting a correct majority is just p. Now suppose we send in the two extra voters. The probability that they change an incorrect majority to a correct majority is (1-p)p², while the probability that they change a correct majority to an incorrect majority is p(1-p)(1-p). The first of these probabilities is greater than the second if and only if p > 1/2, proving the theorem.

Asymptotics

The probability of a correct majority decision P(n,p), when the individual probability p is close to 1/2 grows linearly in terms of p-1/2. For n voters each one having probability p of deciding correctly and for odd n (where there are no possible ties):

$P(n,p)=1/2+c_{1}(p-1/2)+c_{3}(p-1/2)^{3}+O((p-1/2)^{5})$

where

$c_{1}={n \choose {\lfloor n/2\rfloor }}{\frac {\lfloor n/2\rfloor +1}{4^{\lfloor n/2\rfloor }}}={\sqrt {\frac {2n+1}{\pi }}}(1+{\frac {1}{16n^{2}}}+O(n^{-3}))$

and the asymptotic approximation in terms of n is very accurate. The expansion is only in odd powers and $c_{3}<0$. In simple terms, this says that when the decision is difficult (p close to 1/2), the gain by having n voters grows proportionally to ${\sqrt {n}}$.

The non-asymptotic part of Condorcet's jury theorem does not hold for correlated votes

In a jury comprising an odd number of jurors $n$, let $p$ be the probability of a juror voting for the correct alternative and $c$ be the (second-order) correlation coefficient between any two correct votes. If all higher-order correlation coefficients in the Bahadur representation of the joint probability distribution of votes equal to zero, and $(p,c)\in {\mathcal {B}}_{n}$ is an admissible pair, then:

The probability of the jury collectively reaching the correct decision (Condorcet probability) under simple majority is given by:

$P(n,p,c)=I_{p}\left({\frac {n+1}{2}},{\frac {n+1}{2}}\right)+0.5c(n-1)(0.5-p){\frac {\partial I_{p}({\frac {n+1}{2}},{\frac {n+1}{2}})}{\partial p}}$, where $I_{p}$ is the regularized incomplete beta function.

Example: Take a jury of three jurors $(n=3)$, with individual competence $p=0.55$ and second-order correlation $c=0.4$. Then $P(3,0.55,0.4)=0.54505$. The competence of the jury is lower than the competence of a single juror, which equals to $0.55$. Moreover, enlarging the jury by two jurors $(n=5)$ decreases the jury competence $P(5,0.55,0.4)=0.5196194$.

Note that $p=0.55$ and $c=0.4$ is an admissible pair of parameters. For $n=5$ and $p=0.55$, the maximum admissible second-order correlation coefficient equals $\approx 0.43$.

The above example shows that when the individual competence is low but the correlation is high

1. The collective competence under simple majority may fall below that of a single juror,
2. Enlarging the jury may decrease its collective competence.

The above result is due to Kaniovski and Zaigraev, who discuss optimal jury design for homogenous juries with correlated votes.

Limitations

This version of the theorem is correct, given its assumptions, but its assumptions are unrealistic in practice. Some objections that are commonly raised:

• Real votes are not independent, and do not have uniform probabilities. The non-asymptotic part of Condorcet's jury theorem does not hold for correlated votes in general. This is not necessarily a problem since the theorem may still hold under sufficiently general assumptions. One very strong version of the theorem requires only that the average of the individual competence levels of the voters (i.e. the average of their individual probabilities of deciding correctly) is slightly greater than half. This version of the theorem does not require voter independence, but takes into account the degree to which votes may be correlated.
• The notion of "correctness" may not be meaningful when making policy decisions as opposed to deciding questions of fact. Some defenders of the theorem hold that it is applicable when voting is aimed at determining which policy best promotes the public good, rather than at merely expressing individual preferences. On this reading, what the theorem says is that although each member of the electorate may only have a vague perception of which of two policies is better, majority voting has an amplifying effect. The "group competence level", as represented by the probability that the majority chooses the better alternative, increases towards 1 as the size of the electorate grows assuming that each voter is more often right than wrong.
• The theorem doesn't directly apply to decisions between more than two outcomes. This critical limitation was in fact recognized by Condorcet (see Condorcet's paradox), and in general it is very difficult to reconcile individual decisions between three or more outcomes (see Arrow's theorem), although List and Goodin present evidence to the contrary. This limitation may also be overcome by means of a sequence of votes on pairs of alternatives, as is commonly realized via the legislative amendment process. (However, as per Arrow's theorem, this creates a "path dependence" on the exact sequence of pairs of alternatives; e.g., which amendment is proposed first can make a difference in what amendment is ultimately passed, or if the law—with or without amendments—is passed at all.)
• The behaviour that everybody in the jury votes according to his own beliefs might not be a Nash equilibrium under certain circumstances.

Nonetheless, Condorcet's jury theorem provides a theoretical basis for democracy, even if somewhat idealized, as well as a basis of the decision of questions of fact by jury trial, and as such continues to be studied by political scientists.

The theorem in other disciplines

The Condorcet Jury Theorem has recently been used to conceptualize score integration when several physician readers (radiologists, endoscopists, etc.) independently evaluate images for disease activity. This task arises in central reading performed during clinical trials and has similarities to voting. According to the authors, the application of the theorem can translate individual reader scores into a final score in a fashion that is both mathematically sound (by avoiding averaging of ordinal data), mathematically tractable for further analysis, and in a manner that is consistent with the scoring task at hand (based on decisions about the presence or absence of features, a subjective classification task)