Talk:Cramér–Rao bound

From Wikipedia, the free encyclopedia
Jump to: navigation, search
WikiProject Statistics (Rated B-class, Mid-importance)
WikiProject icon

This article is within the scope of the WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page or join the discussion.

B-Class article B  This article has been rated as B-Class on the quality scale.
 Mid  This article has been rated as Mid-importance on the importance scale.
 
WikiProject Mathematics (Rated B-class, Mid-importance)
WikiProject Mathematics
This article is within the scope of WikiProject Mathematics, a collaborative effort to improve the coverage of Mathematics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
Mathematics rating:
B Class
Mid Importance
 Field: Probability and statistics

Lower/Upper bound[edit]

"Upper bound" is correct in the first paragraph, where we speak of "accuracy". The Cramer-Rao inequality gives the maximum accuracy that can be achieved. Later, we speak about variance and there it is in fact a lower bound. High variance means low accuracy and vice versa.

This was changed recently, I have changed it back. -BB

I didn't read carefully the first time, if we're talking about variance then the correct concept is precision not accuracy. Accuracy ties in with unbiasness. I feel we should not bring in accuracy or precision here since traditionally the CRLB is used in direct relation to the variance.
I like it as you have put it now, using precision instead of accuracy. The first sentence now expresses well the basic message of the CRB: It tells you how good any estimator can be, thus limiting the "goodness" by giving its maximum value. -BB

Too Technical[edit]

I have a science/engineering background, but can't begin to understand this. If I could suggest a change, I would. —BenFrantzDale


Hi BenFranz. I'm happy to improve the article. What bit don't you understand? Robinh 14:32, 6 January 2006 (UTC)
For starters, a sentence or two on how this inequality is used (i.e., in what field does it come up) would be helpful. I don't know how specialized this topic is, but I like to think I have most of the background needed to get a rough understanding of it. A list of prerequisites, as described on Wikipedia:Make technical articles accessible would be helpful. —BenFrantzDale 16:25, 6 January 2006 (UTC)

It's used where statistical estimation is used. Read the article on Fisher information. Michael Hardy 22:54, 9 January 2006 (UTC)

a correction for the example for the cramer-rao ineqality[edit]

My name is Roey and I am a student in my third year for industrial engineering in T.A.U, I have a correction for the example given here for cramer rao inequality: the normal distibution formula is incorrect and for some reason I can`t insert it here, in the example the formula has not divided the (x-m)^2 by 2*teta, instead, it is devided by teta. thus the final result is incorrect. The right result should be: 1/2*teta^2 (and not 3 times this number) I have continued this example correctly by I can`t get it to here.... Have a good day and tell me how can I put the correct answer here please...

Roey

Error in Example[edit]

As Roey I found that the example for the Gaussian case was mistaken. I have corrected it. Indeed this example achieves the CR bound. Anyway I'm not too much familiar with this math. I would acknowledge if the original author or other reader will confirm the change.

Viroslav

what log means[edit]

Hi, shouldn't what log means here be defined, or better yet Ln is used instead of log?

MrKeuner

more example suggestions[edit]

Pursuant to the example, it's common to think of the information as the variance of the score (by definition) or the negative expectation of the second derivative of the log likelihood (when it exists); it threw me for a minute to think of the information as the negative expectation of the derivative of the score. In my experience, one generally choses one or the other representation and not a mix of both.

What "log" means for likelihood problems is generally not an issue, (it should be natural log to negate the antilog in the gaussian example) but the standard book notation uses plain "log" with the understanding that it means natural log.

Cramer Rao lower bound is not just for unbiassed estimators[edit]

My understanding is that CRLB exists for biased estimators too. However it is considerably simpler in the unbiased case where the form of the bound does not depend upon the estimator used (it's the same for all unbiased estimators). Might it not be worth stating this somewhere so as not give give the impression that CRLB ONLY exists for unbiased estimators?

MB

86.144.38.235 15:15, 11 April 2007 (UTC)

Sounds like a good idea, go ahead and be bold! --Zvika 12:40, 12 April 2007 (UTC)

Some major changes[edit]

I've made some significant changes to the article. I hope you like them. The main difference is that I changed the order so that the statement of the bound, in its various forms, appears before examples and proofs. This seems to me the correct order for an encyclopedia. I also added a version of the bound for biased estimators, as requested by User:86.144.38.235. Finally, I tried to simplify the mathematical expressions a little bit, especially in the vector case. --Zvika 17:32, 19 May 2007 (UTC)

Aitken and Silverstone had previously discovered this bound. I am thinking of adding a reference as soon as I figure out how to cite it properly. "On the Estimation of Statistical Parameters", Proceedings of the Royal Society of Edinburgh, 1942, vol. 61, pp. 186-194. —Preceding unsigned comment added by 218.101.112.13 (talk) 20:54, 7 November 2008 (UTC)

Error and Clarity Suggestion:

"This in this case, V = derivative of Log PDF..." A) It is unclear whether d/dsigma^2 is the 2nd derivative of sigma or the 1st derivative of sigma squared. B) Taking logs and the derivative in one step is hard for a reader to follow, so break up the calculation in multiple steps. C) (X-mu)^2/2*sigma^2 implies that sigma^2 is in the numerator but sigma^2 belongs in the denominator. CreateW (talk) 23:51, 12 February 2011 (UTC)

redirect from Information inequality[edit]

I find the redirect from Information inequality a bit misleading; I was looking for Inequalities in information theory... - Saibod (talk) 17:37, 20 October 2011 (UTC)

I have replaced the redirect by a disambig page. However, note that presently no articles use a wikilink to Information inequality. Melcombe (talk) 08:47, 21 October 2011 (UTC)
Thanks! I didn't feel "bold" enough to do it myself, so it encourages me to see that someone did it. Is it a problem that no articles use a wikilink to information inequality? - Saibod (talk) 08:46, 26 October 2011 (UTC)
Not particularly, as it provides a route in for anyone searching for the term "information inequality". But it may be worth checking articles for the phrase ""information inequality" to see if there needs to be a wikilink either to one of the aticles or to the disambig page. Melcombe (talk) 08:53, 27 October 2011 (UTC)