Condorcet's jury theorem
Condorcet's jury theorem is a political science theorem about the relative probability of a given group of individuals arriving at a correct decision. The theorem was first expressed by the Marquis de Condorcet in his 1785 work Essay on the Application of Analysis to the Probability of Majority Decisions.
The assumptions of the simplest version of the theorem are that a group wishes to reach a decision by majority vote. One of the two outcomes of the vote is correct, and each voter has an independent probability p of voting for the correct decision. The theorem asks how many voters we should include in the group. The result depends on whether p is greater than or less than 1/2:
- If p is greater than 1/2 (each voter is more likely to vote correctly), then adding more voters increases the probability that the majority decision is correct. In the limit, the probability that the majority votes correctly approaches 1 as the number of voters increases.
- On the other hand, if p is less than 1/2 (each voter is more likely to vote incorrectly), then adding more voters makes things worse: the optimal jury consists of a single voter.
To avoid the need for a tie-breaking rule, we assume n is odd. Essentially the same argument works for even n if ties are broken by fair coin-flips.
Now suppose we start with n voters, and let m of these voters vote correctly.
Consider what happens when we add two more voters (to keep the total number odd). The majority vote changes in only two cases:
- m was one vote too small to get a majority of the n votes, but both new voters voted correctly.
- m was just equal to a majority of the n votes, but both new voters voted incorrectly.
The rest of the time, either the new votes cancel out, only increase the gap, or don't make enough of a difference. So we only care what happens when a single vote (among the first n) separates a correct from an incorrect majority.
Restricting our attention to this case, we can imagine that the first n-1 votes cancel out and that the deciding vote is cast by the n-th voter. In this case the probability of getting a correct majority is just p. Now suppose we send in the two extra voters. The probability that they change an incorrect majority to a correct majority is (1-p)p2, while the probability that they change a correct majority to an incorrect majority is p(1-p)(1-p). The first of these probabilities is greater than the second if and only if p > 1/2, proving the theorem.
This proof is direct; it just sums up the probabilities of the majorities. Each term of the sum multiplies the number of combinations of a majority by the probability of that majority. Each majority is counted using a combination, n items taken k at a time, where n is the jury size, and k is the size of the majority. Probabilities range from 0, the vote is always wrong, to 1, always right. Each person decides independently, so the probabilities of their decisions multiply. The probability of each correct decision is p. The probability of an incorrect decision, q, is the opposite of p, i.e. 1 − p. The power notation, i.e. is a shorthand for x multiplications of p.
Committee or jury accuracies can be easily estimated by using this approach in computer spreadsheets or programs.
First let us take the simplest case of n = 3, p = 0.8. We need to show that 3 people have higher than 0.8 chance of being right. Indeed:
- 0.8 × 0.8 × 0.8 + 0.8 × 0.8 × 0.2 + 0.8 × 0.2 × 0.8 + 0.2 × 0.8 × 0.8 = 0.896.
Now let us take n = 9 (the general case works the same). Following the same considerations as in the n = 3 case, we need to select all options for majorities and multiply by respective p or q = 1 − p. So we need to show:
i.e. we need to show
(reducing denominator by 1 in binomial coefficient)
(reducing nominator and instead multiply by 2)
(swapping p by q < p in last four terms)
(using the polynomial expansion of ).
The probability of a correct majority decision P(n, p), when the individual probability p is close to 1/2 grows linearly in terms of p − 1/2. For n voters each one having probability p of deciding correctly and for odd n (where there are no possible ties):
and the asymptotic approximation in terms of n is very accurate. The expansion is only in odd powers and . In simple terms, this says that when the decision is difficult (p close to 1/2), the gain by having n voters grows proportionally to .
In a jury comprising an odd number of jurors , let be the probability of a juror voting for the correct alternative and be the (second-order) correlation coefficient between any two correct votes. If all higher-order correlation coefficients in the Bahadur representation of the joint probability distribution of votes equal to zero, and is an admissible pair, then:
The probability of the jury collectively reaching the correct decision (Condorcet probability) under simple majority is given by:
where is the regularized incomplete beta function.
Example: Take a jury of three jurors , with individual competence and second-order correlation . Then . The competence of the jury is lower than the competence of a single juror, which equals to . Moreover, enlarging the jury by two jurors decreases the jury competence .
Note that and is an admissible pair of parameters. For and , the maximum admissible second-order correlation coefficient equals .
The above example shows that when the individual competence is low but the correlation is high
- The collective competence under simple majority may fall below that of a single juror,
- Enlarging the jury may decrease its collective competence.
The above result is due to Kaniovski and Zaigraev, who discuss optimal jury design for homogenous juries with correlated votes.
This version of the theorem is correct, given its assumptions, but its assumptions are unrealistic in practice. Some objections that are commonly raised:
- Real votes are not independent, and do not have uniform probabilities. The non-asymptotic part of Condorcet's jury theorem does not hold for correlated votes in general. This is not necessarily a problem since the theorem may still hold under sufficiently general assumptions. One very strong version of the theorem requires only that the average of the individual competence levels of the voters (i.e. the average of their individual probabilities of deciding correctly) is slightly greater than half. This version of the theorem does not require voter independence, but takes into account the degree to which votes may be correlated.
- The notion of "correctness" may not be meaningful when making policy decisions as opposed to deciding questions of fact. Some defenders of the theorem hold that it is applicable when voting is aimed at determining which policy best promotes the public good, rather than at merely expressing individual preferences. On this reading, what the theorem says is that although each member of the electorate may only have a vague perception of which of two policies is better, majority voting has an amplifying effect. The "group competence level", as represented by the probability that the majority chooses the better alternative, increases towards 1 as the size of the electorate grows assuming that each voter is more often right than wrong.
- The theorem doesn't directly apply to decisions between more than two outcomes. This critical limitation was in fact recognized by Condorcet (see Condorcet's paradox), and in general it is very difficult to reconcile individual decisions between three or more outcomes (see Arrow's theorem), although List and Goodin present evidence to the contrary. This limitation may also be overcome by means of a sequence of votes on pairs of alternatives, as is commonly realized via the legislative amendment process. (However, as per Arrow's theorem, this creates a "path dependence" on the exact sequence of pairs of alternatives; e.g., which amendment is proposed first can make a difference in what amendment is ultimately passed, or if the law—with or without amendments—is passed at all.)
- The behaviour that everybody in the jury votes according to his own beliefs might not be a Nash equilibrium under certain circumstances.
Nonetheless, Condorcet's jury theorem provides a theoretical basis for democracy, even if somewhat idealized, as well as a basis of the decision of questions of fact by jury trial, and as such continues to be studied by political scientists.
The theorem in other disciplines
The Condorcet jury theorem has recently been used to conceptualize score integration when several physician readers (radiologists, endoscopists, etc.) independently evaluate images for disease activity. This task arises in central reading performed during clinical trials and has similarities to voting. According to the authors, the application of the theorem can translate individual reader scores into a final score in a fashion that is both mathematically sound (by avoiding averaging of ordinal data), mathematically tractable for further analysis, and in a manner that is consistent with the scoring task at hand (based on decisions about the presence or absence of features, a subjective classification task)
The Condorcet jury theorem is also used in ensemble learning in the field of machine learning. An ensemble method combines the predictions of many individual classifiers by majority voting. Assuming that each of the individual classifiers predict with slightly greater than 50% accuracy, then the ensemble of their predictions will be far greater than their individual predictive scores.
- Marquis de Condorcet (1785). Essai sur l'application de l'analyse à la probabilité des décisions rendues à la pluralité des voix (PNG) (in French). Retrieved 2008-03-10.
- Bahadur, R.R. (1961). "A representation of the joint distribution of responses to n dichotomous items". H. Solomon (ed.), Studies in Item Analysis and Prediction: 158–168.
- Kaniovski, Serguei; Alexander, Zaigraev (2011). "Optimal Jury Design for Homogeneous Juries with Correlated Votes" (PDF). Theory and Decision. 71 (4): 439–459. CiteSeerX 10.1.1.225.5613. doi:10.1007/s11238-009-9170-2.
- see for example: Krishna K. Ladha (August 1992). "The Condorcet Jury Theorem, Free Speech, and Correlated Votes". American Journal of Political Science. 36 (3): 617–634. doi:10.2307/2111584. JSTOR 2111584.
- Bernard Grofman; Guillermo Owen; Scott L. Feld (1983). "Thirteen theorems in search of the truth" (PDF). Theory & Decision. 15 (3): 261–78. doi:10.1007/BF00125672.
- James Hawthorne. "Voting In Search of the Public Good: the Probabilistic Logic of Majority Judgments" (PDF). Retrieved 2009-04-20.
- Christian List and Robert Goodin (September 2001). "Epistemic democracy : generalizing the Condorcet Jury Theorem" (PDF). Journal of Political Philosophy. 9 (3): 277–306. CiteSeerX 10.1.1.105.9476. doi:10.1111/1467-9760.00128.
- Austen-Smith, David; Banks, Jeffrey S. (1996). "Information aggregation, rationality, and the Condorcet Jury Theorem". American Political Science Review. 90 (1): 34–45. doi:10.2307/2082796. JSTOR 2082796.
- Gottlieb, Klaus; Hussain, Fez (2015-02-19). "Voting for Image Scoring and Assessment (VISA) - theory and application of a 2 + 1 reader algorithm to improve accuracy of imaging endpoints in clinical trials". BMC Medical Imaging. 15: 6. doi:10.1186/s12880-015-0049-0. ISSN 1471-2342. PMC 4349725. PMID 25880066.