# Maximum likelihood sequence estimation

Maximum likelihood sequence estimation (MLSE) is a mathematical algorithm to extract useful data out of a noisy data stream.

## Theory

For an optimized detector for digital signals the priority is not to reconstruct the transmitter signal, but it should do a best estimation of the transmitted data with the least possible number of errors. The receiver emulates the distorted channel. All possible transmitted data streams are fed into this distorted channel model. The receiver compares the time response with the actual received signal and determines the most likely signal. In cases that are most computationally straightforward, root mean square deviation can be used as the decision criterion[1] for the lowest error probability.

## Background

Suppose that there is an underlying signal {x(t)}, of which an observed signal {r(t)} is available. The observed signal r is related to x via a transformation that may be nonlinear and may involve attenuation, and would usually involve the incorporation of random noise. The statistical parameters of this transformation are assumed known. The problem to be solved is to use the observations {r(t)} to create a good estimate of {x(t)}.

Maximum likelihood sequence estimation is formally the application of maximum likelihood to this problem. That is, the estimate of {x(t)} is defined to be sequence of values which maximize the functional

$L(x)=p(r|x),$

where p(r|x) denotes the conditional joint probability density function of the observed series {r(t)} given that the underlying series has the values {x(t)}.

In contrast, the related method of maximum a posteriori estimation is formally the application of the Maximum a posteriori (MAP) estimation approach. This is more complex than maximum likelihood sequence estimation and requires a known distribution (in Bayesian terms, a prior distribution) for the underlying signal. In this case the estimate of {x(t)} is defined to be sequence of values which maximize the functional

$P(x)=p(x|r),$

where p(x|r) denotes the conditional joint probability density function of the underlying series {x(t)} given that the observed series has taken the values {r(t)}. Bayes' theorem implies that

$P(x)=p(x|r)=\frac{p(r|x)p(x)}{p(r)}.$

In cases where the contribution of random noise is additive and has a multivariate normal distribution, the problem of maximum likelihood sequence estimation can be reduced to that of a least squares minimization.

## References

1. ^ G. Bosco, P. Poggiolini, and M. Visintin, "Performance Analysis of MLSE Receivers Based on the Square-Root Metric," J. Lightwave Technol. 26, 2098–2109 (2008)