Rectified Gaussian distribution

In probability theory, the rectified Gaussian distribution is a modification of the Gaussian distribution when its negative elements are reset to 0 (analogous to an electronic rectifier). It is essentially a mixture of a discrete distribution (constant 0) and a continuous distribution (a truncated Gaussian distribution with interval $(0,\infty )$ ) as a result of censoring.

Density function

The probability density function of a rectified Gaussian distribution, for which random variables X having this distribution, derived from the normal distribution ${\mathcal {N}}(\mu ,\sigma ^{2}),$ are displayed as $X\sim {\mathcal {N}}^{\textrm {R}}(\mu ,\sigma ^{2})$ , is given by

$f(x;\mu ,\sigma ^{2})=\Phi (-{\frac {\mu }{\sigma }})\delta (x)+{\frac {1}{\sqrt {2\pi \sigma ^{2}}}}\;e^{-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}}{\textrm {U}}(x).$  A comparison of Gaussian distribution, rectified Gaussian distribution, and truncated Gaussian distribution.

Here, $\Phi (x)$ is the cumulative distribution function (cdf) of the standard normal distribution:

$\Phi (x)={\frac {1}{\sqrt {2\pi }}}\int _{-\infty }^{x}e^{-t^{2}/2}\,dt\quad x\in \mathbb {R} ,$ $\delta (x)$ is the Dirac delta function

$\delta (x)={\begin{cases}+\infty ,&x=0\\0,&x\neq 0\end{cases}}$ and, ${\textrm {U}}(x)$ is the unit step function:

${\textrm {U}}(x)={\begin{cases}0,&x\leq 0,\\1,&x>0.\end{cases}}$ Mean and variance

Since the unrectified normal distribution has mean $\mu$ and since in transforming it to the rectified distribution some probability mass has been shifted to a higher value (from negative values to 0), the mean of the rectified distribution is greater than $\mu .$ Since the rectified distribution is formed by moving some of the probability mass toward the rest of the probability mass, the rectification is a mean-preserving contraction combined with a mean-changing rigid shift of the distribution, and thus the variance is decreased; therefore the variance of the rectified distribution is less than $\sigma ^{2}.$ Generating values

To generate values computationally, one can use

$s\sim {\mathcal {N}}(\mu ,\sigma ^{2}),\quad x={\textrm {max}}(0,s),$ and then

$x\sim {\mathcal {N}}^{\textrm {R}}(\mu ,\sigma ^{2}).$ Application

A rectified Gaussian distribution is semi-conjugate to the Gaussian likelihood, and it has been recently applied to factor analysis, or particularly, (non-negative) rectified factor analysis. Harva  proposed a variational learning algorithm for the rectified factor model, where the factors follow a mixture of rectified Gaussian; and later Meng  proposed an infinite rectified factor model coupled with its Gibbs sampling solution, where the factors follow a Dirichlet process mixture of rectified Gaussian distribution, and applied it in computational biology for reconstruction of gene regulatory networks.