Bernstein–von Mises theorem: Difference between revisions
No edit summary |
Aaron McDaid (talk | contribs) A couple of limitation with Freedman's result. That result doesn't apply to the priors used in the real world. |
||
| Line 2: | Line 2: | ||
In [[Bayesian inference]], the '''Bernstein–von Mises theorem''' provides the basis for the important result that the [[posterior distribution]] for unknown quantities in any problem is effectively independent of the [[prior distribution]] (assuming it obeys [[Cromwell's rule]]) once the amount of information supplied by a sample of data is large enough. <ref>{{cite book | last=van der Vaart|first=A.W.|title=Asymptotic Statistics |year=1998|publisher=Cambridge University Press|isbn=0-521-78450-6|chapter=10.2 Bernstein–von Mises Theorem}}</ref> |
In [[Bayesian inference]], the '''Bernstein–von Mises theorem''' provides the basis for the important result that the [[posterior distribution]] for unknown quantities in any problem is effectively independent of the [[prior distribution]] (assuming it obeys [[Cromwell's rule]]) once the amount of information supplied by a sample of data is large enough. <ref>{{cite book | last=van der Vaart|first=A.W.|title=Asymptotic Statistics |year=1998|publisher=Cambridge University Press|isbn=0-521-78450-6|chapter=10.2 Bernstein–von Mises Theorem}}</ref> |
||
The theorem is named after [[Richard von Mises]] and [[S. N. Bernstein]] even though the first proper proof was given by [[Joseph L. Doob]] in 1949 for random variables with finite [[probability space]].<ref>{{cite journal |last1=Doob |first1=Joseph L. |year=1949 |title=Applications of the theory of martingales |journal=Colloq. Intern. du C.N.R.S (Paris) |pages=22–28|volume=13}}</ref> Later [[Lucien Le Cam]], his PhD student [[Lorraine Schwarz]], [[David A. Freedman (statistician)|David A. Freedman]] and [[Persi Diaconis]] extended the proof under more general assumptions. |
The theorem is named after [[Richard von Mises]] and [[S. N. Bernstein]] even though the first proper proof was given by [[Joseph L. Doob]] in 1949 for random variables with finite [[probability space]].<ref>{{cite journal |last1=Doob |first1=Joseph L. |year=1949 |title=Applications of the theory of martingales |journal=Colloq. Intern. du C.N.R.S (Paris) |pages=22–28|volume=13}}</ref> Later [[Lucien Le Cam]], his PhD student [[Lorraine Schwarz]], [[David A. Freedman (statistician)|David A. Freedman]] and [[Persi Diaconis]] extended the proof under more general assumptions. |
||
=== limitations === |
|||
| ⚫ | The statistician [[A. W. F. Edwards]] has remarked, "It is sometimes said, in defence of the Bayesian concept, that the choice of prior distribution is unimportant in practice, because it hardly influences the posterior distribution at all when there are moderate amounts of data. The less said about this 'defence' the better."<ref>{{cite book | last = Edwards | first = A.W.F. | title = Likelihood | publisher = Johns Hopkins University Press | location = Baltimore | year = 1992 | isbn = 0-8018-4443-6 }}</ref> |
||
A remarkable result was found by Freedman in 1965: the Bernstein–von Mises theorem does not hold [[almost surely]] if the random variable has an infinite countable [[probability space]]; however this depends on allowing a very broad range of possible priors. In practice, the priors used typically in research do have the desirable property even with an infinite countable [[probability space]]. |
|||
Also, it is important to draw a distinction between the ''posterior mode'' and other summaries of the posterior, such as its mean. Under Freedman's examples, the posterior density (including its mean, for example) can converge on the wrong result, but it should be noted that the posterior mode is consistent and will converge on the correct result. |
|||
| ⚫ | The statistician [[A. W. F. Edwards]] has remarked, "It is sometimes said, in defence of the Bayesian concept, that the choice of prior distribution is unimportant in practice, because it hardly influences the posterior distribution at all when there are moderate amounts of data. The less said about this 'defence' the better."<ref>{{cite book | last = Edwards | first = A.W.F. | title = Likelihood | publisher = Johns Hopkins University Press | location = Baltimore | year = 1992 | isbn = 0-8018-4443-6 }}</ref> This criticism does not apply to the posterior mode. |
||
==Notes== |
==Notes== |
||
Revision as of 20:51, 23 May 2013
 | This article includes a list of general references, but it lacks sufficient corresponding inline citations. (October 2011) |
In Bayesian inference, the Bernstein–von Mises theorem provides the basis for the important result that the posterior distribution for unknown quantities in any problem is effectively independent of the prior distribution (assuming it obeys Cromwell's rule) once the amount of information supplied by a sample of data is large enough. [1]
The theorem is named after Richard von Mises and S. N. Bernstein even though the first proper proof was given by Joseph L. Doob in 1949 for random variables with finite probability space.[2] Later Lucien Le Cam, his PhD student Lorraine Schwarz, David A. Freedman and Persi Diaconis extended the proof under more general assumptions.
limitations
A remarkable result was found by Freedman in 1965: the Bernstein–von Mises theorem does not hold almost surely if the random variable has an infinite countable probability space; however this depends on allowing a very broad range of possible priors. In practice, the priors used typically in research do have the desirable property even with an infinite countable probability space.
Also, it is important to draw a distinction between the posterior mode and other summaries of the posterior, such as its mean. Under Freedman's examples, the posterior density (including its mean, for example) can converge on the wrong result, but it should be noted that the posterior mode is consistent and will converge on the correct result.
The statistician A. W. F. Edwards has remarked, "It is sometimes said, in defence of the Bayesian concept, that the choice of prior distribution is unimportant in practice, because it hardly influences the posterior distribution at all when there are moderate amounts of data. The less said about this 'defence' the better."[3] This criticism does not apply to the posterior mode.
Notes
- ^ van der Vaart, A.W. (1998). "10.2 Bernstein–von Mises Theorem". Asymptotic Statistics. Cambridge University Press. ISBN 0-521-78450-6.
- ^ Doob, Joseph L. (1949). "Applications of the theory of martingales". Colloq. Intern. du C.N.R.S (Paris). 13: 22–28.
- ^ Edwards, A.W.F. (1992). Likelihood. Baltimore: Johns Hopkins University Press. ISBN 0-8018-4443-6.
References
- van der Vaart, A.W. (1998). "10.2 Bernstein–von Mises Theorem". Asymptotic Statistics. Cambridge University Press. ISBN 0-521-78450-6.
- Doob, Joseph L. (1949), “Applications of the theory of martingales”. Colloq. Intern. du C.N.R.S (Paris), No. 13, pp. 22–28.
- Freedman, David A. (1963), “On the asymptotic behaviour of behaviour of Bayes estimates in the discrete case I”. The Annals of Mathematical Statistics, vol. 34, pp. 1386–1403.
- Freedman, David A. (1965), “On the asymptotic behaviour of behaviour of Bayes estimates in the discrete case II”. The Annals of Mathematical Statistics, vol. 36, pp. 454–456.
- Le Cam, Lucien (1986) Asymptotic Methods in Statistical Decision Theory, Springer. ISBN 0-387-96307-3 (Pages 336 and 618–621).
- Lorraine Schwartz (1965), “On Bayes procedure”. Z. Wahrscheinlichkeitstheorie, No. 4, pp. 10–26.
 | This statistics-related article is a stub. You can help Wikipedia by expanding it. |