Talk:Bayesian probability

Objections to Laplace's "Objective" priors (by Fisher and others)

All of the criticisms of Laplace (with which I am familiar) object to "objective" priors (Principle of insufficient reason).

Laplace also used non-uniform priors sometimes. Can anybody find criticisms of all uses of "non-objective" priors by Fisher?

[Fisher's words are reverently quoted (despite the notorious unreliability of statistician's "history" and especially Fisher's scholarship, sic.) but Fisher's quoted objections seem to apply to what Fisher (with characteristic obscurity) called "axiomatic priors"?]

Should not mention be made of Laplace's informed priors? (If agreeable, then I can look at Hald's serious volume through 1930, or Stigler).

(These questions are indeed part of my "agenda" to reduce the false dichotomy of objective and subjective Bayes methods, I confess!) Kiefer.Wolfowitz (talk) 14:17, 21 June 2009 (UTC)

There is no "false dichotomy": there are many references that discuss and oppose these two versions of Bayesian probability (see many Jaynes references for example). That difference may not be there (according to you that is indeed the case, it seems), but the references surely are there. What can be done, is to find a reference that clearly states that this difference is a "false dicotomy" or similar according to someone. Original research will not do! Tomixdf (talk) 14:59, 21 June 2009 (UTC)

What do you think of the substantive & editorical questions? (I gave a parenthetical confession to avoid being "outed" again---or at least to save you typing . . . . )Kiefer.Wolfowitz (talk) 15:24, 21 June 2009 (UTC)

Here's a fine example. Gelman wrote a provocative article, in which a hypothetical anti-Bayesian argues against Bayesian methods. Bernardo answers, stating that he answers from the "objective Bayesian" POV. Joseph Kadane states he answers from the subjective POV. So false or not, the distinction is out there. I don't know what you mean by "substantive & editorical questions", "parenthetical confession" or "outed". Tomixdf (talk) 07:47, 22 June 2009 (UTC)

I repeat my questions:

1. Can anybody find criticisms of all uses of "non-objective" priors by Fisher? (Again, Fisher used informed priors.)
2. Should not mention be made of Laplace's informed priors?

Kiefer.Wolfowitz (talk) 12:06, 22 June 2009 (UTC)

What exactly is your point? Tomixdf (talk) 12:17, 22 June 2009 (UTC)

Empirical Bayes

This article cites Turing, Good, etc. as being Bayesian.

Please do not remove statements about "Empirical Bayes" here without substantial justification; such justification could entail removing Turing and Good also. Kiefer.Wolfowitz (talk) 18:47, 21 June 2009 (UTC)

This article is about Bayesian probability.

Good and Turing belong in this article because they clearly identified their philosophical view of probability as classical/Bayesian; not because they may sometimes have used Empirical Bayes.

"Empirical Bayes" isn't a method based on Bayesian probabilities. It's a method based on frequentist probabilities. (Which is why Neyman liked it). Yes, it can be regarded as an approximation to a fully Bayesian method. But along the way everything which requires a distinctively Bayesian view of probability is dropped.

It doesn't require a Bayesian "degree of belief" interpretation of the hidden parameters θ. Rather, the θs are seen as a number of samples from a large population - a fully frequentist conception, allowing them to be seen as frequentist random variables. As for the hyperparameters φ, a Bayesian would draw up priors for them. But EB doesn't. Instead it treats them as frequentist parameters of the distribution of the random variables θ, and estimates them in a frequentist think-of-an-estimator way.

So that's why, if you look at eg the Gelman Bayesian Analysis "Objections" paper and its follow-ups, it's quite amusing just how at pains the Bayesians are to emphasise EB not being a "Bayesian" method. Jheald (talk) 19:48, 21 June 2009 (UTC)

I agree with most of what you write, and we could have both found similar citations from Dennis Lindley, who has written that Empirical Bayes is the non-Bayesian method around! (I have been told.)

Nonetheless, Good and others have used EB as part of the Bayes/Berkeley compromise, which is necessary in statistical practice. Who can be a subjectivist Bayes with more than a few parameters? But Good is a much more serious statistician and philosopher than Gelman (who writes about "noninformative priors" without embarrassment), and so I would ask that we take Good at his own word, that EB is a useful Bayesian method. (I trust that I don't need to find a quote from Good Thinking!) Kiefer.Wolfowitz (talk) 20:21, 21 June 2009 (UTC)

The point, though, is that it doesn't require any belief in "Bayesian probability" to use and believe in EB. So it is misleading if the article could be construed to imply, even potentially, that a readiness to accept EB suggests any support for the Bayesian idea of probability. Jheald (talk) 20:39, 21 June 2009 (UTC)

Does not your Empirical Bayes warn the faithful against the infidels, Savage and de Finetti? Is that not warning enough? (Smiley face)

What I wrote was that Neyman used (the?) Bayesian formalism repeatedly, and considered EB great. Delete the link to EB at the bottom of the page (under see also), if you want, but don't touch my lines on Neyman! Please! Is that okay? Kiefer.Wolfowitz (talk) 20:50, 21 June 2009 (UTC)

But this article isn't about the Bayesian formalism. That has its own article. This article is about the Bayesian view of what probability is. Jheald (talk) 21:14, 21 June 2009 (UTC)

I agree with Jheald. What is the point of mentioning that Neyman was fond of EB (a frequentist method)? Or that Fisher and Neyman sporadically used methods that could be viewed as Bayesian? The point is that they both promoted and adhered to a very different view on probability. Kiefer.Wolfowitz's contributions only serve to obscure this fact. Tomixdf (talk) 06:16, 22 June 2009 (UTC)

In the Neyman reference that was provided in the EB sentence, Neyman writes "A brilliant idea as to how it can be done, initiating a novel chapter of frequentist mathematical statistics, is due to Herbert Robbins." That settles it, surely: Neyman saw EB as frequentist (which it also is). I removed the EB sentence. Tomixdf (talk) 13:48, 22 June 2009 (UTC)

Neyman as Bayesian

The article states: "Neyman also used Bayesian methods in teaching[12] and sometimes in research[13]." I glanced through [12], and it is a firm defense of the frequentist view. There is no mentioning of Neyman using Bayesian methods. [13] is a technical paper by Neyman; thus the conclusion that "Neyman also used Bayesian methods in research" is Original Research, and not up to Wikipedia standards. If no better references can be provided, the sentence needs to be removed. Tomixdf (talk) 14:00, 22 June 2009 (UTC)

I'll rephrase the disputed sentence later. However, your objectionable heading for this section misrepresents my sentence, about Neyman's use of Bayesian methods.

The article (12) ends (from memory) with explaining how Churchill suggested using the sampling distribution of an estimator, after Neyman taught Bayes with a uniform prior, so your synopsis of 12 is wrong. By Tarski's convention T, the sentence "Neyman used Bayesian methods in research" is true if Neyman used Bayesian methods in research; hence the relevance of [13], which is technical (like most statistical articles, you might as well learn now). Kiefer.Wolfowitz (talk) 14:41, 22 June 2009 (UTC)

(12) ends with Neyman thanking Churchill Eisenhart for suggesting that "the whole theory would be nicer if it were built from the start without any reference to Bayesianism and priors". He then writes "This remark proved inspiring". (Not from memory; the article is in front of me) That is exactly the opposite of what you try to use this reference for. Tomixdf (talk) 14:54, 22 June 2009 (UTC)

Tomixdf, please confirm that Neyman describes teaching inverse probability (Bayesian methods) in that article (which was the proposition I wrote, and cited that article as the most convenient and well known example).Kiefer.Wolfowitz (talk) 15:57, 22 June 2009 (UTC)

The article does NOT state "Neyman uses inverse probability (Bayesian methods) in teaching". Please stop abusing references to push your POVs. Tomixdf (talk) 16:01, 22 June 2009 (UTC)

QUOTE: "Neyman also used Bayesian methods in teaching" is what I wrote. What is your problem (here)? Please apologize. Kiefer.Wolfowitz (talk) 16:08, 22 June 2009 (UTC)

My problem is the CONTINUOUS abuse of references to push your POV. I dare you to cite a section in the article that states that "Neyman also used Bayesian methods in teaching". Tomixdf (talk) 16:11, 22 June 2009 (UTC)

This is ridiculous. Neyman differs from the Queen of England in not referring to himself in the third person. Your "dare" is irrelevant. Kiefer.Wolfowitz (talk) 16:16, 22 June 2009 (UTC)

Fine. Then provide a section that states "I also used Bayesian methods in teaching". 16:19, 22 June 2009 (UTC)

I know that you are an inexperienced Wikipedia editor (judged by your removing text on the talk page, including jokes in articles, personal insults and innuendo, spamming the talk page), so please take the time to read up on original research. You need a reference that specifically states "Person A is a Bayesian", not "I think Person A is a Bayesian because something in article X by Person A makes me think so". That is OR and not acceptable in Wikipedia. Tomixdf (talk) 15:02, 22 June 2009 (UTC)

I never included a joke in the article. What innuendo are you talking about? Kiefer.Wolfowitz (talk) 15:57, 22 June 2009 (UTC)

You included a link to "problem gambling" as a joke, and even tried to defend it afterwards. Beginners make mistakes, and that's understandable, but at a certain point it should start to improve (quod non). Tomixdf (talk) 16:14, 22 June 2009 (UTC)

That was not a joke, although it was funny imho. Why don't you write van Fraassen and ask him to assess the text I wrote? Kiefer.Wolfowitz (talk) 16:19, 22 June 2009 (UTC)

Why on earth would I do that? You need to read up on Wikipedia practice. Tomixdf (talk) 16:22, 22 June 2009 (UTC)

Scientific Method

This section seems a little biased to a non-Bayesian like me. It says that scientific practice (or good scientific practice) can be interpreted as an application of Bayesianism. Surely this is controversial. It also implied that the criticisms of Bayesianism would warrant further examination if there were concrete examples of Bayesianism going wrong. Again, this seems too much of an opinion for an encyclopedia entry. One might think that the problems are as serious in the abstract as they are in the concrete. I've at least corrected this, so that it reads 'might' rather than 'would.'

It now read that scientific method "is sometimes interpreted" as an application of Bayesian updating---the existing footnote refers to two standard sources. Kiefer.Wolfowitz (talk) 18:41, 17 September 2009 (UTC)

I propose adding a link in external references

The proposed link is http://www.opentradingsystem.com/quantNotes/Conditional_probability_.html

It contains examples of calculations using Bayesian formula.

Does anyone object?