Tweedie distribution

In probability and statistics, the Tweedie distributions are a family of probability distributions which include such as the normal and gamma, the purely discrete scaled Poisson distribution, and the class of mixed compound Poisson-Gamma distributions which have positive mass at zero, but are otherwise continuous. For any random variable Y that obeys a Tweedie distribution the variance var(Y) relates to the mean E(Y) by the power law,

${\text{var}}\,(Y)=a{\text{E}}\,(Y)^{p}$,

where a and p are positive constants.

The Tweedie distributions were named by Bent Jørgensen after Maurice Charles Kenneth Tweedie, a Statistician and medical physicist at the University of Liverpool, UK, who presented the first thorough study of these distributions in 1984.

Definitions

Tweedie distributions are a special case of exponential dispersion models, a class of models used to describe error distributions for the generalized linear model. The term exponential dispersion model refers to the exponential form that these models take, evident from the canonical equation used to describe the distribution P_λ,θ of the random variable Z on the measurable sets A,

$P_{\lambda ,\theta }(Z\in A)=\int _{A}\exp[\theta \cdot z-\lambda \kappa (\theta )]\cdot \nu _{\lambda }\,(dz)$,

with the interrelated measures ν_λ. θ is the canonical parameter; the cumulant function is

$\kappa (\theta )=\lambda ^{-1}\log \int e^{\theta z}\cdot \nu _{\lambda }\,(dz)$;

λ is the index parameter; and z the canonical statistic. This equation represents a family of exponential dispersion models ED^*(θ,λ) that are completely determined by the parameters θ and λ and the cumulant function.

Additive exponential dispersion models

The models just described are additive models with the property that the distribution of the sum of independent random variables,

$Z_{+}=Z_{1}+\ldots +Z_{n}$,

for which Z_i~ED^*(θ,λ_i) with fixed θ and various λ are members of the family of distributions with the same θ,

$Z_{+}\sim ED^{*}(\theta ,\lambda _{1}+\ldots +\lambda _{n})$.

Reproductive exponential dispersion models

A second class of exponential dispersion models exists designated by the random variable

$Y=Z/\lambda \sim ED(\mu ,\sigma ^{2})$,

where σ²=1/λ, known as reproductive exponential dispersion models. They have the property that for n independent random variables Y_i~ED(μ,σ²/w_i), with weighting factors w_i and

$w=\sum _{i=1}^{n}w_{i}$,

a weighted average of the variables gives,

$w^{-1}\sum _{i=1}^{n}w_{i}Y_{i}\sim ED(\mu ,\sigma ^{2}/w)$.

For reproductive models the weighted average of independent random variables with fixed μ and σ² and various values for w_i is a member of the family of distributions with same μ and σ².

The Tweedie exponential dispersion models are both additive and reproductive; we thus have the duality transformation

$Y\mapsto Z=Y/\sigma ^{2}$.

Scale invariance

A third property of the Tweedie models is that they are scale invariant: For a reproductive exponential dispersion model ED(μ,σ²) and any positive constant c we have the property of closure under scale transformation,

$cED(\mu ,\sigma ^{2})=ED(c\mu ,c^{2-p}\sigma ^{2})$,

where the index parameter p is a real-valued unitless constant. With this transformation the new variable Y’=cY belongs to the family of distributions with fixed μ and σ² but different values of c.

The Tweedie power variance function

To define the variance function for exponential dispersion models we make use of the mean value mapping, the relationship between the canonical parameter θ and the mean μ. It is define by the function

$\tau (\theta )=\kappa ^{\prime }(\theta )=\mu$.

The variance function V(μ) is constructed from the mean value mapping,

$V(\mu )=\tau ^{\prime }[\tau ^{-1}(\mu )]$.

Here the minus exponent in τ ^-1(μ) denotes an inverse function rather than a reciprocal. The mean and variance of an additive random variable is then E(Z)=λμ and var(Z)=λV(μ).

Scale invariance implies that the variance function obeys the relationship V(μ)=μ ^p.

The Tweedie cumulant generating functions

The properties of exponential dispersion models give us two differential equations. The first relates the mean value mapping and the variance function to each other,

${\frac {\partial \tau ^{-1}(\mu )}{\partial \mu }}={\frac {1}{V(\mu )}}$.

The second shows how the mean value mapping is related to the cumulant function,

${\frac {\partial \kappa (\theta )}{\partial \theta }}=\tau (\theta )$.

These equations can be solved to obtain the cumulant function for different cases of the Tweedie models. A cumulant generating function (CGF) may then be obtained from the cumulant function. The additive CGF is generally specified by the equation

$K^{*}(s)=\log[{\text{E}}(e^{sZ})]=\lambda [\kappa (\theta +s)-\kappa (\theta )]$,

and the reproductive CGF by

$K(s)=\log[{\text{E}}(e^{sY})]=\lambda [\kappa (\theta +s/\lambda )-\kappa (\theta )]$,

where s is the generating function variable.

The cumulant functions for specific values of the index parameter p are

$\kappa _{p}(\theta )={\begin{cases}{\dfrac {\alpha -1}{\alpha }}\left({\dfrac {\theta }{\alpha -1}}\right)^{\alpha }&\quad p\neq 1,2\\-\log(-\theta )&\quad p=2\\e^{\theta }&\quad p=1\end{cases}}$,

where α is the Tweedie exponent

$\alpha ={\dfrac {p-2}{p-1}}$.

For the additive Tweedie models the CGFs take the form,

$K_{p}^{*}(s;\theta ,\lambda )={\begin{cases}\lambda \kappa _{p}(\theta )[(1+s/\theta )^{\alpha }-1]&\quad p\neq 1,2\\-\lambda \log(1+s/\theta )&\quad p=2\\\lambda e^{\theta }(e^{s}-1)&\quad p=1\end{cases}}$,

and for the reproductive models,

$K_{p}(s;\theta ,\lambda )={\begin{cases}\lambda \kappa _{p}(\theta ){[1+s/(\theta \lambda )]^{\alpha }-1}&\quad p\neq 1,2\\-\lambda \log[1+s/(\theta \lambda )]&\quad p=2\\\lambda e^{\theta }(e^{s/\lambda }-1)&\quad p=1\end{cases}}$.

The additive and reproductive Tweedie models are conventionally denoted by the symbols Tw^*_p(θ,λ) and Tw_p(θ,σ²), respectively.

The Tweedie distributions include a number of familiar distributions as well as some unusual ones, each being specified by the domain of the index parameter. We have the

• normal distribution, p=0,

• Poisson distribution, p=1,

• compound Poisson-gamma distribution, 1<p<2,

• gamma distribution, p=2,

• positive stable distributions, 2<p<3,

• inverse Gaussian distribution, p=3,

• positive stable distributions, p>3, and

• extreme stable distributions, p=∞.

• For 0<p<1 no Tweedie model exists.

The first and second derivatives of the CGFs, with s=0, yields the mean and variance, respectively. One can thus confirm that for the additive models the variance relates to the mean by the power law,

$\mathrm {var} (Z)\propto \mathrm {E} (Z)^{p}$.

The Tweedie convergence theorem

The Tweedie exponential dispersion models are fundamental in statistical theory consequent to their roles as foci of convergence for a wide range of statistical processes. Jørgensen et al proved a theorem that specifies the asymptotic behaviour of variance functions known as the Tweedie convergence theorem This theorem, in technical terms, is stated thus: The unit variance function is regular of order p at zero (or infinity) provided that V(μ)~c₀μ^p for μ as it approaches zero (or infinity) for all real values of p and c₀ >0. Then for a unit variance function regular of order p at either zero or infinity and for

$p\notin (0,1)$ ,

for any $\mu >0$, and $\sigma ^{2}>0$ we have

$c^{-1}ED(c\mu ,\sigma ^{2}c^{2-p})\rightarrow Tw_{p}(\mu ,c_{0}\sigma ^{2})$

as $c\downarrow 0$ or $c\rightarrow \infty$, respectively, where the convergence is through values of c such that cμ is in the domain of θ and c^p-2/σ² is in the domain of λ. The model must be infinitely divisible as c^2-p approaches infinity.

In nontechnical terms this theorem implies that any exponential dispersion model that asymptotically manifests a variance to mean power law is required to have a variance function that comes within the domain of attraction of a Tweedie model. Almost all distribution functions with finite cumulant generating functions qualify as exponential dispersion models and most exponential dispersion models manifest variance functions of this form. Hence many probability distributions have variance functions that express this asymptotic behavior, and the Tweedie distributions become foci of convergence for a wide range of data types.

The Tweedie models and Taylor’s power law

Taylor's power law is an empirical law in ecology that relates the variance of the number of individuals of a species per unit area of habitat to the corresponding mean by a power law relationship. For the population count Y with mean µ and variance var(Y), Taylor’s law is written,

${\text{var}}\,(Y)=a\mu ^{p}$,

where a and p are both positive constants. Since L R Taylor described this law in 1961 there have been many different explanations offered to explain it, ranging from animal behavior, a random walk model, a [[birth-death process|stochastic birth, death, immigration and emigration model], to a consequence of equilibrium and non-equilibrium statistical mechanics. No consensus exists as to an explanation for this model.

Since Taylor’s law is mathematically identical to the variance to mean power law that characterizes the Tweedie models, it seemed reasonable to use these models and the Tweedie convergence theorem to explain the observed clustering of animals and plants associated with Taylor’s law. The majority of the observed values for the power law exponent p have fallen in the interval (1,2) and so the Tweedie compound Poisson-gamma distribution would seem particularly applicable. Comparison of the empirical distribution function to the theoretical compound Poisson-gamma distribution has provided a means to verify consistency of this hypothesis.

Whereas conventional models for Taylor’s law have tended to involve ad hoc animal behavioral or population dynamic assumptions, the Tweedie convergence theorem would imply that Taylor’s law results from a general mathematical convergence effect much as how the central limit theorem governs the convergence behavior of certain types of random data. Indeed, any mathematical model, approximation or simulation that is designed to yield Taylor’s law (on the basis of this theorem) is required to converge to the form of the Tweedie models.

The double power law

The eponym Taylor's power law has been applied to a wide range of data that manifests a variance to mean power function. However, subtle mathematical differences exist between the transformational properties of some of these data. A double power law, which includes Taylor’s original law, has been proposed to describe these differences. For a population count drawn from an area of size t with mean abundance per unit area µ, and where

${\text{E}}\,[Y(\mu ;t)]=t\mu \,\!$,

we have for the double power law:

${\text{var}}\,[Y(\mu ;t)]=a\mu ^{p}t^{2-d}\,\!$.

The constant d∈ [0,1] has been identified as a fractal exponent. The factor aµ ^p represents the original Taylor’s law, a function of the mean abundance per unit area; the last term t^2-d describes how the power law scales with the enumerative bin size. This last term implies a statistically self-similar scaling of the spatial distribution of items of interest as the size of the bin changes. It is this second portion of the double power law that underlies the variance to mean power law reported in systems like regional blood flow heterogeneity, the genomic distribution of SNPs and genes, and number theoretic examples.

Tweedie convergence and 1/f noise

1/f noise (also known as pink noise) refers to a pattern of noise characterized by a power law relationship between its intensity S(f) at different frequencies f,

$S(f)\propto 1/f^{\gamma }$,

where the dimensionless exponent γ∈ [0,1]. It is found within a diverse number of natural processes. . Many different explanations for 1/f noise exist, a widely-held hypothesis is based on Self-organized criticality where dynamical systems close to a critical point are thought to manifest scale-invariant spatial and/or temporal behavior.

In this subsection a mathematical connection between 1/f noise and the Tweedie variance to mean power law will be described. To begin, we first need to introduce self-similar processes: For the sequence of numbers

$Y=(Y_{i}:i=0,1,2,...,N)$

with mean

${\hat {\mu }}={\text{E}}(Y_{i})$,

deviations

$y_{i}=Y_{i}-{\hat {\mu }}$,

variance

${\hat {\sigma }}^{2}={\text{E}}(y_{i}^{2})$,

and autocorrelation function

$r(k)={\text{E}}(y_{i},y_{i+k})/{\text{E}}(y_{i}^{2})$

with lag k, if the autocorrelation of this sequence has the long range behavior

$r(k)\sim k^{-d}L(k)$

as k→∞ and where L(k) is a slowly varying function at large values of k, this sequence is called a self-similar process.

The method of expanding bins can be used to analyze self-similar processes. Consider a set of equal-sized non-overlapping bins that divides the original sequence of N elements into groups of m equal-sized segments (N/m is integer) so that new reproductive sequences, based on the mean values, can be defined:

$Y_{i}^{(m)}=(Y_{im-m+1}+...+Y_{im})/m$.

The variance determined from this sequence will scale as the bin size changes such that

${\text{var}}[Y^{(m)}]={\hat {\sigma }}^{2}m^{-d}$

if and only if the autocorrelation has the limiting form

$\lim _{k\to \infty }r(k)/k^{-d}=(2-d)(1-d)/2$.

One can also construct a set of corresponding additive sequences

$Z_{i}^{(m)}=mY_{i}^{(m)}$,

based on the expanding bins,

$Z_{i}^{(m)}=(Y_{im-m+1}+...+Y_{im})$.

Provided the autocorrelation function exhibits the same behavior, the additive sequences will obey the relationship

${\text{var}}[Z_{i}^{(m)}]=m^{2}{\text{var}}[Y^{(m)}]=({\hat {\sigma }}^{2}/{\hat {\mu }}^{2-d}){\text{E}}[Z_{i}^{(m)}]^{2-d}$

Since ${\hat {\mu }}$ and ${\hat {\sigma }}^{2}$ are constants this relationship constitutes a variance to mean power law, with p=2-d.

The biconditional relationship above between the variance to mean power law and power law autocorrelation function, and the Wiener–Khinchin theorem imply that any sequence that exhibits a variance to mean power law by the method of expanding bins will also manifest 1/f noise, and vice versa. Moreover, the Tweedie convergence theorem, by virtue of its central limit-like effect of generating distributions that manifest variance to mean power functions, will also generate processes that manifest 1/f noise. The Tweedie convergence theorem thus allows provides an alternative explanation for the origin of 1/f noise, based its central limit-like effect.

Much as the central limit theorem requires certain kinds of random processes to have as a focus of their convergence the Gaussian distribution and thus express white noise, the Tweedie convergence requires certain non-Gaussian processes to have as a focus of convergence the Tweedie distributions that express 1/f noise.

The Tweedie models and multifractality

From the properties of self-similar processes, the power law exponent p=2-d is related to the Hurst exponent H and the fractal dimension D by

$D=2-H=2-p/2$.

A one-dimensional data sequence of self-similar data may demonstrate a variance to mean power law with local variations in the value of p and hence in the value of D. When fractal structures manifest local variations in fractal dimension, they are said to be multifractals. Examples of data sequences that exhibit local variations in p like this include the eigenvalue deviations of theGaussian Orthogonal and Unitary Ensembles. The Tweedie compound Poisson-gamma distribution has served to model multifractality based on local variations in the Tweedie exponent α. Consequently, in conjunction with the variation of α, the Tweedie convergence theorem can be viewed as having a role in the genesis of such multifractals.

Applications

Regional organ blood flow

Regional organ blood flow has been traditionally assessed by the injection of radiolabelled polyethylene microspheres into the arterial circulation of animals, of a size that they become entrapped within the microcirculation of organs. The organ to be assessed is then divided into equal-sized cubes and the amount of radiolabel within each cube is evaluated by liquid scintillation counting and recorded. The amount of radioactivity within each cube is taken to reflect the blood flow through that sample at the time of injection. It is possible to evaluate adjacent cubes from an organ in order to additively determine the blood flow through larger regions. Through the work of J B Bassingthwaighte and others an empirical power law has been derived between the relative dispersion of blood flow of tissue samples (RD=standard deviation/ mean)of mass m relative to reference sized samples:

$RD(m)=RD(m_{ref})\left({\frac {m}{m_{ref}}}\right)^{1-D_{s}}$

This power law exponent D_s has been called a fractal dimension. Bassingthwaighte’s power law can be shown to directly relate to the variance to mean power law. Regional organ blood flow can thus be modelled by the Tweedie compound Poisson-gamma distribution. In this model tissue sample could be considered to contain a random (Poisson) distributed number of entrapment sites, each with gamma distributed blood flow. Blood flow at this microcirculatory level has been observed to obey a gamma distribution, thus providing support for this hypothesis.

Cancer metastasis

The experimental cancer metastasis assay has some resemblance to the above method to measure regional blood flow. Groups of syngeneic and age matched mice are given intravenous injections of equal-sized aliquots of suspensions of cloned cancer cells and then after a set period of time their lungs are removed and the number of cancer metastases enumerated within each pair of lungs. If other groups of mice are injected with different cancer cell clones then the number of metastases per group will differ in accordance with the metastatic potentials of the clones. It has been long recognized that there can be considerable intraclonal variation in the numbers of metastases per mouse despite the best attempts to keep the experimental conditions within each clonal group uniform. This variation is larger than would be expected on the basis of a Poisson distribution of numbers of metastases per mouse in each clone and when the variance of the number of metastases per mouse was plotted against the corresponding mean a power law was found.

The variance to mean power law for metastases was found to also hold for spontaneous murine metastases and for cases series of human metastases. Since hematogenous metastasis occurs in direct relationship to regional blood flow and videomicroscopic studies indicate that the passage and entrapment of cancer cells within the circulation appears analogous to the microsphere experiments it seemed plausible to propose that the variation in numbers of hematogenous metastases could reflect heterogeneity in regional organ blood flow. The blood flow model was based on the Tweedie compound Poisson-gamma distribution, a distribution governing a continuous random variable. For that reason in the metastasis model it was assumed that blood flow was governed by that distribution and that the number of regional metastases occurred as a Poisson process for which the intensity was directly proportional to blood flow. This lead to the description of the Poisson negative binomial (PNB) distribution as a discrete equivalent to the Tweedie compound Poisson-gamma distribution. The probability generating function for the PNB distribution is

$G(s)=\exp \left[\lambda {\frac {\alpha -1}{\alpha }}\left({\frac {\theta }{\alpha -1}}\right)^{\alpha }\left\{\left(1-{\frac {1}{\theta }}+{\frac {s}{\theta }}\right)^{\alpha }-1\right\}\right]$.

The relationship between the mean and variance of the PNB distribution is then

${\text{var}}\,(Y)=a{\text{E}}(Y)^{b}+{\text{E}}(Y)$,

which, in the range of many experimental metastasis assays, would be indistinguishable from the variance to mean power law. For sparse data, however, this discrete variance to mean relationship would behave more like that of a Poisson distribution where the variance equaled the mean.

Genomic structure and evolution

The local density of Single Nucleotide Polymorphisms (SNPs) within the human genome, as well as that of genes, appears to cluster in accord with the variance to mean power law and the Tweedie compound Poisson-gamma distribution. In the case of SNPs their observed density reflects the assessment techniques, the availability of genomic sequences for analysis, and the nucleotide heterozygosity. The first two factors reflect ascertainment errors inherent to the collection methods, the latter factor reflects an intrinsic property of the genome.

In the coalescent model of population genetics each genetic locus has its own unique history. Within the evolution of a population from some species some genetic loci could presumably be traced back to a relatively recent common ancestor whereas other loci might have more ancient genealogies. More ancient genomic segments would have had more time to accumulate SNPs and to experience recombination. R R Hudson has proposed a model where recombination could cause variation in the time to most common recent ancestor for different genomic segments. A high recombination rate could cause a chromosome to contain a large number of small segments with less correlated genealogies.

Assuming a constant background rate of mutation the number of SNPs per genomic segment would accumulate proportionately to the time to the most recent common ancestor. Current population genetic theory would indicate that these times would be gamma distributed, on average. The Tweedie compound Poisson-gamma distribution would suggest a model whereby the SNP map would consist of multiple small genomic segments with the mean number of SNPs per segment would be gamma distributed as per Hudson’s model.

The distribution of genes within the human genome also demonstrated a variance to mean power law, when the method of expanding bins was used to determine the corresponding variances and means. Similarly the number of genes per enumerative bin was found to obey a Tweedie compound Poisson-gamma distribution. This probability distribution was deemed compatible with two different biological models: the microarrangement model where the number of genes per unit genomic length was determined by the sum of a random number of smaller genomic segments derived by random breakage and reconstruction of protochormosomes. These smaller segments would be assumed to carry on average a gamma distributed number of genes.

In the alternative gene cluster model, genes would be distributed randomly within the protochromosomes. Over large evolutionary timescales there would occur tandem duplication, mutations, insertions, deletions and rearrangements that could affect the genes through a stochastic birth, death and immigration process to yield the Tweedie compound Poisson-gamma distribution.

Both these mechanisms would implicate neutral evolutionary processes that would result in regional clustering of genes.

Random matrix theory

The Gaussian unitary ensemble (GUE) consists of complex Hermitian matrices that are invariant under unitary transformations whereas the Gaussian orthogonal ensemble (GOE) consists of real symmetric matrices invariant under orthogonal transformations. The ranked eigenvalues E_n from these random matrices obey Wigner’s semicircular distribution: For a NxN matrix the average density for eigenvalues of size E will be

${\bar {\rho }}(E)={\begin{cases}{\sqrt {2N-E^{2}}}/\pi &\quad \left\vert E\right\vert <{\sqrt {2N}}\\0&\quad \left\vert E\right\vert >{\sqrt {2N}}\end{cases}}$

as E→ ∞ . Integration of the semicircular rule provides the number of eigenvalues on average less than E,

${\bar {\eta }}(E)={\frac {1}{2\pi }}\left[E{\sqrt {2N-E^{2}}}+2N\arcsin \left({\frac {E}{\sqrt {2N}}}\right)+\pi N\right]$.

The ranked eigenvalues can be unfolded, or renormalized, with the equation

$e_{n}={\bar {\eta }}(E)=\int \limits _{-\infty }^{E_{n}}dE^{\prime }{\bar {\rho }}(E^{\prime })$.

This removes the trend of the sequence from the fluctuating portion. If we look at the absolute value of the difference between the actual and expected cumulative number of eigenvalues

$\left|{\bar {D}}_{n}\right|=\left|n-{\bar {\eta }}(E_{n})\right|$

we obtain a sequence of eigenvalue fluctuations which, using the method of expanding bins, reveals a variance to mean power law. The eigenvalue fluctuations of both the GUE and the GOE manifest this power law with the power law exponents ranging between 1 and 2, and they similarly manifest 1/f noise spectra. These eigenvalue fluctuations also correspond to the Tweedie compound Poisson-gamma distribution and they exhibit multifractality.

Other applications

Applications of Tweedie distributions include:

• actuarial studies
• assay analysis
• survival analysis
• ecology
• analysis of alcohol consumption in British teenagers
• medical applications
• meteorology and climatology
• fisheries
• Mertens function

Further reading

• Kaas, R. (2005). "Compound Poisson distribution and GLM’s – Tweedie’s distribution". In Proceedings of the Contact Forum "3rd Actuarial and Financial Mathematics Day", pages 3–12. Brussels: Royal Flemish Academy of Belgium for Science and the Arts.
• Ohlsson, E and Johansson, B. (2003) Exact Credibility and Tweedie Models, University of Stockholm, Research report, October 2003.
• Tweedie, M.C.K. (1956). "Some statistical properties of inverse Gaussian distributions". Virginia J. Sci. (N.S.) 7, 160—165.

External links

• Tweedie distributions. http://www.statsci.org/s/tweedie.html
• Tweedie generalized linear model family. http://www.statsci.org/s/tweedief.html
• Examples of use of the model. http://www.sci.usq.edu.au/staff/dunn/Datasets/tech-glms.html#Tweedie
• tweeDEseq: R package for RNA-seq data analysis using the Poisson-Tweedie family of distributions. http://bioconductor.org/packages/2.9/bioc/html/tweeDEseq.html