From Wikipedia, the free encyclopedia
In probability theory , the law of total covariance ,[1] covariance decomposition formula , or ECCE states that if X , Y , and Z are random variables on the same probability space , and the covariance of X and Y is finite, then
cov
(
X
,
Y
)
=
E
(
cov
(
X
,
Y
∣
Z
)
)
+
cov
(
E
(
X
∣
Z
)
,
E
(
Y
∣
Z
)
)
.
{\displaystyle \operatorname {cov} (X,Y)=\operatorname {E} (\operatorname {cov} (X,Y\mid Z))+\operatorname {cov} (\operatorname {E} (X\mid Z),\operatorname {E} (Y\mid Z)).\,}
The nomenclature in this article's title parallels the phrase law of total variance [2]
(The conditional expected values E( X | Z ) and E( Y | Z ) are random variables in their own right, whose values depends on the value of Z . Notice that the conditional expected value of X given the event Z = z is a function of z (this is where adherence to the conventional rigidly case-sensitive notation of probability theory becomes important!). If we write E( X | Z = z ) = g (z ) then the random variable E( X | Z ) is just g (Z ). Similar comments apply to the conditional covariance.)
Proof The law of total covariance can be proved using the law of total expectation : First,
cov
[
X
,
Y
]
=
E
[
X
Y
]
−
E
[
X
]
E
[
Y
]
{\displaystyle \operatorname {cov} [X,Y]=\operatorname {E} [XY]-\operatorname {E} [X]\operatorname {E} [Y]}
from the definition of covariance. Then we apply the law of total expectation by conditioning on the random variable Z :
=
E
[
E
[
X
Y
∣
Z
]
]
−
E
[
E
[
X
∣
Z
]
]
E
[
E
[
Y
∣
Z
]
]
{\displaystyle =\operatorname {E} [\operatorname {E} [XY\mid Z]]-\operatorname {E} [\operatorname {E} [X\mid Z]]\operatorname {E} [\operatorname {E} [Y\mid Z]]}
Now we rewrite the term inside the first expectation using the definition of covariance:
=
E
[
cov
[
X
,
Y
∣
Z
]
+
E
[
X
∣
Z
]
E
[
Y
∣
Z
]
]
−
E
[
E
[
X
∣
Z
]
]
E
[
E
[
Y
∣
Z
]
]
{\displaystyle =\operatorname {E} \!\left[\operatorname {cov} [X,Y\mid Z]+\operatorname {E} [X\mid Z]\operatorname {E} [Y\mid Z]\right]-\operatorname {E} [\operatorname {E} [X\mid Z]]\operatorname {E} [\operatorname {E} [Y\mid Z]]}
Since expectation of a sum is the sum of expectations, we can regroup the terms:
=
E
[
cov
[
X
,
Y
∣
Z
]
]
+
E
[
E
[
X
∣
Z
]
E
[
Y
∣
Z
]
]
−
E
[
E
[
X
∣
Z
]
]
E
[
E
[
Y
∣
Z
]
]
{\displaystyle =\operatorname {E} \!\left[\operatorname {cov} [X,Y\mid Z]]+\operatorname {E} [\operatorname {E} [X\mid Z]\operatorname {E} [Y\mid Z]\right]-\operatorname {E} [\operatorname {E} [X\mid Z]]\operatorname {E} [\operatorname {E} [Y\mid Z]]}
Finally, we recognize the final two terms as the covariance of the conditional expectations E[X |Z ] and E[Y |Z ]:
=
E
(
cov
(
X
,
Y
∣
Z
)
)
+
cov
(
E
(
X
∣
Z
)
,
E
(
Y
∣
Z
)
)
{\displaystyle =\operatorname {E} (\operatorname {cov} (X,Y\mid Z))+\operatorname {cov} (\operatorname {E} (X\mid Z),\operatorname {E} (Y\mid Z))}
Notes and references
^ Matthew R. Rudary, On Predictive Linear Gaussian Models , ProQuest, 2009, page 121.
^ Sheldon M. Ross, A First Course in Probability , sixth edition, Prentice Hall, 2002, page 392.
See also External links 
![{\displaystyle \operatorname {cov} [X,Y]=\operatorname {E} [XY]-\operatorname {E} [X]\operatorname {E} [Y]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/ded56b10a565c8b572e646b2814a031b94770228)
![{\displaystyle =\operatorname {E} [\operatorname {E} [XY\mid Z]]-\operatorname {E} [\operatorname {E} [X\mid Z]]\operatorname {E} [\operatorname {E} [Y\mid Z]]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f058892015ea186389cfe6671d52c442d4ba0398)
![{\displaystyle =\operatorname {E} \!\left[\operatorname {cov} [X,Y\mid Z]+\operatorname {E} [X\mid Z]\operatorname {E} [Y\mid Z]\right]-\operatorname {E} [\operatorname {E} [X\mid Z]]\operatorname {E} [\operatorname {E} [Y\mid Z]]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/b835c3995e71f7872005b964380d46a404def898)
![{\displaystyle =\operatorname {E} \!\left[\operatorname {cov} [X,Y\mid Z]]+\operatorname {E} [\operatorname {E} [X\mid Z]\operatorname {E} [Y\mid Z]\right]-\operatorname {E} [\operatorname {E} [X\mid Z]]\operatorname {E} [\operatorname {E} [Y\mid Z]]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/effd760443a907a7690018f33a2cb39cbed1a6e2)
