Scoring algorithm, also known as Fisher's scoring, is a form of Newton's method used in statistics to solve maximum likelihood equations numerically, named after Ronald Fisher.
Sketch of Derivation
Let be random variables, independent and identically distributed with twice differentiable p.d.f. , and we wish to calculate the maximum likelihood estimator (M.L.E.) of . First, suppose we have a starting point for our algorithm , and consider a Taylor expansion of the score function, , about :
is the observed information matrix at . Now, setting , using that and rearranging gives us:
We therefore use the algorithm
and under certain regularity conditions, it can be shown that .
In practice, is usually replaced by , the Fisher information, thus giving us the Fisher Scoring Algorithm:
Jennrich, R. I., & Sampson, P. F. (1976). Newton-Raphson and related algorithms for maximum likelihood variance component estimation. Technometrics, 18, 11-17.