Condorcet's jury theorem
Condorcet's jury theorem is a political science theorem about the relative probability of a given group of individuals arriving at a correct decision. The theorem was first expressed by the Marquis de Condorcet in his 1785 work Essay on the Application of Analysis to the Probability of Majority Decisions.
The assumptions of the simplest version of the theorem are that a group wishes to reach a decision by majority vote. One of the two outcomes of the vote is correct, and each voter has an independent probability p of voting for the correct decision. The theorem asks how many voters we should include in the group. The result depends on whether p is greater than or less than 1/2:
- If p is greater than 1/2 (each voter is more likely to vote correctly), then adding more voters increases the probability that the majority decision is correct. In the limit, the probability that the majority votes correctly approaches 1 as the number of voters increases.
- On the other hand, if p is less than 1/2 (each voter is more likely to vote incorrectly), then adding more voters makes things worse: the optimal jury consists of a single voter.
Proof 1: Calculating the probability that two additional voters change the outcome
To avoid the need for a tie-breaking rule, we assume n is odd. Essentially the same argument works for even n if ties are broken by fair coin-flips.
Now suppose we start with n voters, and let m of these voters vote correctly.
Consider what happens when we add two more voters (to keep the total number odd). The majority vote changes in only two cases:
- m was one vote too small to get a majority of the n votes, but both new voters voted correctly.
- m was just equal to a majority of the n votes, but both new voters voted incorrectly.
The rest of the time, either the new votes cancel out, only increase the gap, or don't make enough of a difference. So we only care what happens when a single vote (among the first n) separates a correct from an incorrect majority.
Restricting our attention to this case, we can imagine that the first n-1 votes cancel out and that the deciding vote is cast by the n-th voter. In this case the probability of getting a correct majority is just p. Now suppose we send in the two extra voters. The probability that they change an incorrect majority to a correct majority is (1-p)p2, while the probability that they change a correct majority to an incorrect majority is p(1-p)(1-p). The first of these probabilities is greater than the second if and only if p > 1/2, proving the theorem.
Proof 2: Calculating the probability that the decision is correct
This proof is direct; it just sums up the probabilities of the majorities. Each term of the sum multiplies the number of combinations of a majority by the probability of that majority. Each majority is counted using a combination, n items taken k at a time, where n is the jury size, and k is the size of the majority. Probabilities range from 0, the vote is always wrong, to 1, always right. Each person decides independently, so the probabilities of their decisions multiply. The probability of each correct decision is p. The probability of an incorrect decision, q, is the opposite of p, i.e. 1 − p. The power notation, i.e. is a shorthand for x multiplications of p.
Committee or jury accuracies can be easily estimated by using this approach in computer spreadsheets or programs.
First let us take the simplest case of n = 3, p = 0.8. We need to show that 3 people have higher than 0.8 chance of being right. Indeed:
- 0.8 × 0.8 × 0.8 + 0.8 × 0.8 × 0.2 + 0.8 × 0.2 × 0.8 + 0.2 × 0.8 × 0.8 = 0.896.
This section does not cite any sources. (March 2020) (Learn how and when to remove this template message)
The probability of a correct majority decision P(n, p), when the individual probability p is close to 1/2 grows linearly in terms of p − 1/2. For n voters each one having probability p of deciding correctly and for odd n (where there are no possible ties):
and the asymptotic approximation in terms of n is very accurate. The expansion is only in odd powers and . In simple terms, this says that when the decision is difficult (p close to 1/2), the gain by having n voters grows proportionally to .
Condorcet's theorem assumes that all voters have the same competence, i.e., the probability of deciding correctly is uniform among all voters. In practice, different voters have different competence levels.
A stronger version of the theorem requires only that the average of the individual competence levels of the voters (i.e. the average of their individual probabilities of deciding correctly) is slightly greater than half.
Condorcet's theorem assumes that the votes are statistically independent. But real votes are not independent: voters are often influenced by other voters, causing a peer pressure effect.
The non-asymptotic part of Condorcet's jury theorem does not hold for correlated votes in general. This is not necessarily a problem since the theorem may still hold under sufficiently general assumptions. A strong version of the theorem does not require voter independence, but takes into account the degree to which votes may be correlated.
In a jury comprising an odd number of jurors , let be the probability of a juror voting for the correct alternative and be the (second-order) correlation coefficient between any two correct votes. If all higher-order correlation coefficients in the Bahadur representation of the joint probability distribution of votes equal to zero, and is an admissible pair, then:
The probability of the jury collectively reaching the correct decision (Condorcet probability) under simple majority is given by:
where is the regularized incomplete beta function.
Example: Take a jury of three jurors , with individual competence and second-order correlation . Then . The competence of the jury is lower than the competence of a single juror, which equals to . Moreover, enlarging the jury by two jurors decreases the jury competence .
Note that and is an admissible pair of parameters. For and , the maximum admissible second-order correlation coefficient equals .
The above example shows that when the individual competence is low but the correlation is high
- The collective competence under simple majority may fall below that of a single juror,
- Enlarging the jury may decrease its collective competence.
The above result is due to Kaniovski and Zaigraev, who discuss optimal jury design for homogenous juries with correlated votes.
Indirect majority systems
Condorcet's theorem considers a direct majority system, in which all votes are counted directly towards the final outcome. Many countries use an indirect majority system, in which the voters are divided into groups. The voters in each group decide on an outcome by an internal majority vote; then, the groups decide on the final outcome by a majority vote among them. For example, suppose there are 15 voters. In a direct majority system, a decision is accepted whenever at least 8 votes support it. Suppose now that the voters are grouped into 3 groups of size 5 each. A decision is accepted whenever at least 2 groups support it, and in each group, a decision is accepted whenever at least 3 voters support it. Therefore, a decision may be accepted even if only 6 voters support it.
Boland, Proschan and Tong prove that, when the voters are independent and p>1/2, a direct majority system - as in Condorcet's theorem - always has a higher chance of accepting the correct decision than any indirect majority system.
Berg and Paroush consider multi-tier voting hierarchies, which may have several levels with different decision-making rules in each level. They study the optimal voting structure, and compares the competence against the benefit of time-saving and other expenses.
Condorcet's theorem is correct given its assumptions, but its assumptions are unrealistic in practice. Besides the issue of correlated votes, some objections that are commonly raised are:
1. The notion of "correctness" may not be meaningful when making policy decisions, as opposed to deciding questions of fact. Some defenders of the theorem hold that it is applicable when voting is aimed at determining which policy best promotes the public good, rather than at merely expressing individual preferences. On this reading, what the theorem says is that although each member of the electorate may only have a vague perception of which of two policies is better, majority voting has an amplifying effect. The "group competence level", as represented by the probability that the majority chooses the better alternative, increases towards 1 as the size of the electorate grows assuming that each voter is more often right than wrong.
2. The theorem doesn't directly apply to decisions between more than two outcomes. This critical limitation was in fact recognized by Condorcet (see Condorcet's paradox), and in general it is very difficult to reconcile individual decisions between three or more outcomes (see Arrow's theorem), although List and Goodin present evidence to the contrary. This limitation may also be overcome by means of a sequence of votes on pairs of alternatives, as is commonly realized via the legislative amendment process. (However, as per Arrow's theorem, this creates a "path dependence" on the exact sequence of pairs of alternatives; e.g., which amendment is proposed first can make a difference in what amendment is ultimately passed, or if the law—with or without amendments—is passed at all.)
Despite these objections, Condorcet's jury theorem provides a theoretical basis for democracy, even if somewhat idealized, as well as a basis of the decision of questions of fact by jury trial, and as such continues to be studied by political scientists.
The theorem in other disciplines
The Condorcet jury theorem has recently been used to conceptualize score integration when several physician readers (radiologists, endoscopists, etc.) independently evaluate images for disease activity. This task arises in central reading performed during clinical trials and has similarities to voting. According to the authors, the application of the theorem can translate individual reader scores into a final score in a fashion that is both mathematically sound (by avoiding averaging of ordinal data), mathematically tractable for further analysis, and in a manner that is consistent with the scoring task at hand (based on decisions about the presence or absence of features, a subjective classification task)
The Condorcet jury theorem is also used in ensemble learning in the field of machine learning. An ensemble method combines the predictions of many individual classifiers by majority voting. Assuming that each of the individual classifiers predict with slightly greater than 50% accuracy and their predictions are independent, then the ensemble of their predictions will be far greater than their individual predictive scores.
- Non-asymptotic Condorcet jury theorem.
- Majority systems and the Condorcet jury theorem: discusses non-homogeneous and correlated voters, and indirect majority systems.
- Evolution in collective decision making.
- Marquis de Condorcet (1785). Essai sur l'application de l'analyse à la probabilité des décisions rendues à la pluralité des voix (PNG) (in French). Retrieved 2008-03-10. CS1 maint: discouraged parameter (link)
- Tangian, Andranik (2020). Analytical theory of democracy. Vols. 1 and 2. Cham, Switzerland: Springer. pp. 149–162. ISBN 978-3-030-39690-9.
- Bernard Grofman; Guillermo Owen; Scott L. Feld (1983). "Thirteen theorems in search of the truth" (PDF). Theory & Decision. 15 (3): 261–78. doi:10.1007/BF00125672.
- Kaniovski, Serguei; Alexander, Zaigraev (2011). "Optimal Jury Design for Homogeneous Juries with Correlated Votes" (PDF). Theory and Decision. 71 (4): 439–459. CiteSeerX 10.1.1.225.5613. doi:10.1007/s11238-009-9170-2.
- see for example: Krishna K. Ladha (August 1992). "The Condorcet Jury Theorem, Free Speech, and Correlated Votes". American Journal of Political Science. 36 (3): 617–634. doi:10.2307/2111584. JSTOR 2111584.
- James Hawthorne. "Voting In Search of the Public Good: the Probabilistic Logic of Majority Judgments" (PDF). Archived from the original (PDF) on 2016-03-23. Retrieved 2009-04-20. CS1 maint: discouraged parameter (link)
- Bahadur, R.R. (1961). "A representation of the joint distribution of responses to n dichotomous items". H. Solomon (Ed.), Studies in Item Analysis and Prediction: 158–168.
- Boland, Philip J. (1989). "Majority Systems and the Condorcet Jury Theorem". Journal of the Royal Statistical Society, Series D (The Statistician). 38 (3): 181–189. doi:10.2307/2348873. ISSN 1467-9884. JSTOR 2348873.
- Boland, Philip J.; Proschan, Frank; Tong, Y. L. (March 1989). "Modelling dependence in simple and indirect majority systems". Journal of Applied Probability. 26 (1): 81–88. doi:10.2307/3214318. ISSN 0021-9002. JSTOR 3214318.
- Berg, Sven; Paroush, Jacob (1998-05-01). "Collective decision making in hierarchies". Mathematical Social Sciences. 35 (3): 233–244. doi:10.1016/S0165-4896(97)00047-4. ISSN 0165-4896.
- Christian List and Robert Goodin (September 2001). "Epistemic democracy : generalizing the Condorcet Jury Theorem" (PDF). Journal of Political Philosophy. 9 (3): 277–306. CiteSeerX 10.1.1.105.9476. doi:10.1111/1467-9760.00128.
- Austen-Smith, David; Banks, Jeffrey S. (1996). "Information aggregation, rationality, and the Condorcet Jury Theorem" (PDF). American Political Science Review. 90 (1): 34–45. doi:10.2307/2082796. JSTOR 2082796.
- Gottlieb, Klaus; Hussain, Fez (2015-02-19). "Voting for Image Scoring and Assessment (VISA) - theory and application of a 2 + 1 reader algorithm to improve accuracy of imaging endpoints in clinical trials". BMC Medical Imaging. 15: 6. doi:10.1186/s12880-015-0049-0. ISSN 1471-2342. PMC 4349725. PMID 25880066.
- Ben-Yashar, Ruth; Paroush, Jacob (2000-03-01). "A nonasymptotic Condorcet jury theorem". Social Choice and Welfare. 17 (2): 189–199. doi:10.1007/s003550050014. ISSN 1432-217X.
- "Evolution in collective decision making". Understanding Collective Decision Making: 167–192. 2017. doi:10.4337/9781783473151.00011. ISBN 9781783473151.