Talk:Rao–Blackwell theorem

From Wikipedia, the free encyclopedia
Jump to: navigation, search
WikiProject Statistics (Rated B-class, High-importance)
WikiProject icon

This article is within the scope of the WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page or join the discussion.

B-Class article B  This article has been rated as B-Class on the quality scale.
 High  This article has been rated as High-importance on the importance scale.
 
Cscr-former.svg Rao–Blackwell theorem is a former featured article candidate. Please view the links under Article milestones below to see why the nomination failed. For older candidates, please check the archive.
March 1, 2004 Featured article candidate Not promoted

Lehmann-Scheffé minimum variance[edit]

In the article states that if the NEW estimator is complete and sufficient then it is the minimum variance. But doesn't the Lehmann-Scheffé deal specifically with using a complete and sufficent statistic to find a new estimator given an unbiased estimator? ZioX 22:51, 21 March 2007 (UTC)

Looks as if it ought to say if the statistic on which you condition is complete and sufficient, and the estimator you start with is unbiased, then the Rao-Blackwell estimator is the best unbiased estimator. Michael Hardy 22:37, 21 March 2007 (UTC)
Yes, that's what I figured. I didn't want to change without saying anything. ZioX 22:51, 21 March 2007 (UTC)
Changed it. ZioX 21:05, 22 March 2007 (UTC)

Example[edit]

Calculating delta_1 is not as trivial as it's being made out to be. At least not to the casual reader. Perhaps something should be said about X_1|sum(X_i) ~ Bin(sum(X_i),1/n)? ZioX 22:56, 21 March 2007 (UTC)


I think correct delta_1 is (1-Sn/n) instead of written (1-1/n)^Sn. Will anyone confirm it? K.Y. 27 November 2008 —Preceding unsigned comment added by 119.240.60.195 (talk) 16:29, 26 November 2008 (UTC)

You're mistaken; see details below. Michael Hardy (talk) 17:07, 26 November 2008 (UTC)

Example[edit]

Phone calls arrive at a switchboard according to a Poisson process at an average rate of λ per minute. This rate is not observable, but the numbers X1, ..., Xn of phone calls that arrived during n successive one-minute periods are observed. It is desired to estimate the probability e−λ that the next one-minute period passes with no phone calls.

An extremely crude estimator of the desired probability is

\delta_0=\left\{\begin{matrix}1 & \mbox{if}\ X_1=0 \\
0 & \mbox{otherwise}\end{matrix}\right\},

i.e., this estimates this probability to be 1 if no phone calls arrived in the first minute and zero otherwise. Despite the apparent limitations of this estimator, the result given by its Rao–Blackwellization is a very good estimator.

The sum

 S_n = \sum_{i=1}^n X_{i} = X_1+\cdots+X_n\,\!

can be readily shown to be a sufficient statistic for λ, i.e., the conditional distribution of the data X1, ..., Xn, given this sum, does not depend on λ. Therefore, we find the Rao–Blackwell estimator

\delta_1=\operatorname{E}(\delta_0|S_n).\,\!

After doing some algebra we have

\delta_1=\left(1-{1 \over n}\right)^{S_n}.\,\!

Algebraic details[edit]

We want to evaluate the conditional expected value

 \delta_1=\operatorname{E}(\delta_0|S_n).\,

So let us consider


\begin{align}
\delta_1 & = \operatorname{E}(\delta_0|S_n = s) = \Pr(X_1 = 0 \mid S_n = s) \\  \\
& = \frac{ \Pr(X_1 = 0 \text{ and } X_1 + \cdots + X_n = s) }{\Pr(S_n = s)} \\  \\
& = \frac{ \Pr(X_1 = 0 \text{ and } X_2 + \cdots + X_n = s) }{\Pr(S_n = s)} \\  \\
& = \frac{ \Pr(X_1 = 0) \Pr(X_2 + \cdots + X_n = s)}{\Pr(S_n = s)} \\  \\
& = \frac{\Pr(\operatorname{Poisson}(\lambda)=0) \cdot \Pr(\operatorname{Poisson}((n-1)\lambda) = s)}{\Pr(\operatorname{Poisson}(n\lambda) = s)} \\  \\
& = \frac{ e^{-\lambda} \cdot [((n-1)\lambda)^s e^{-(n-1)\lambda}/s!]}{(n\lambda)^s e^{-n\lambda}/s!} \\  \\
& = \left(\frac{n-1}{n}\right)^s = \left(1 - \frac{1}{n}\right)^s.
\end{align}

And so,

 \operatorname{E}(\delta_1 \mid S_n) = \left(1 - \frac{1}{n}\right)^{S_n}.

Michael Hardy (talk) 17:03, 26 November 2008 (UTC)


Thank you very much for your quick response. And the detail is quite clear.

I tried to get this conditional expectation with an binary process on pi:=P(X_1=0|pi)=exp(-lambda). Now I wondering why I got (1-Sn/n) by deleting parameter with

Combin(n-1, s)*pi^(n-1-s)*(1-pi)^s * pi


.

Combin(n-1, s)*pi^(n-1-s)*(1-pi)^s * pi + Combin(n-1, s-1)*pi^(n-1-(s-1))*(1-pi)^(s-1) * (1-pi)

Is this substituting (Poisson to Binary) harmful? Now I don't have to adhere it, but I want to confirm.

If you have any comments, please tell me. Thank you in advance.

K.Y. to Michael Hardy —Preceding unsigned comment added by 119.240.60.195 (talk) 19:35, 26 November 2008 (UTC)

mistakes[edit]

Sufficiency:

The article states that "a sufficient statistic T(X) for a parameter k is a statistic such that the conditional distribution of the data X, given T(X), depends only on parameter k and not on any other parameters". However, it is the other way around: The conditional distribution does NOT depend on the parameter since T explains it "sufficiently".


Completeness/ idempotence:

T is complete and sufficient does not imply idempotence of the Rao-Blackwell-precedure, as stated in the article.

For example, consider Gaussian random variables X_1,...,X_n with unknown expectation \mu and known variance. Then, it is widely known that the mean of the random variables is sufficient and complete. However, if we consider the median of X_1,...,X_n as an estimator of the expectation, the corresponding Rao-Blackwell-estimator will simply be the mean. Thus, the precedure is not idempotent.

The only way, it stays the same is that the original estimator is measurable with respect to the sigma-algebra induced by the sufficient statistic. —Preceding unsigned comment added by 82.113.106.4 (talk) 00:13, 20 December 2010 (UTC)

References[edit]

This article has a message saying that it needs citations. I found some sources, but I'm unsure how reliable they are. One is the transcript of a lecture at University of California, San Diego, and the other is a professor's notes for students from Georgia Institute of Technology. Which, if any, should I include? Weux082690 (talk) 00:28, 29 April 2013 (UTC)