Transfer entropy

Transfer entropy is a non-parametric statistic measuring the amount of directed (time-asymmetric) transfer of information between two random processes. Transfer entropy from a process X to another process Y is the amount of uncertainty reduced in future values of Y by knowing the past values of X given past values of Y. More specifically, if $X_{t}$ and $Y_{t}$ for $t\in \mathbb {N}$ denote two random processes and the amount of information is measured using Shannon's entropy, the transfer entropy can be written as:

$T_{X\rightarrow Y}=H\left(Y^{t}\mid Y^{\mathbf {t-1} }\right)-H\left(Y^{t}\mid Y^{\mathbf {t-1} },X^{\mathbf {t-1} }\right),$

where H(X) is Shannon entropy of X. The above definition of transfer entropy has been extended by other types of entropy measures such as Rényi entropy.

Transfer entropy reduces to Granger causality for vector auto-regressive processes. Hence, it is advantageous when the model assumption of Granger causality doesn't hold, for e.g. analysis of non-linear signals. However, it is currently limited to the bivariate analysis and usually requires more samples for accurate estimation.

Transfer entropy has been used for estimation of functional connectivity of neurons and social influence in social networks.

See also

• Causality
• Causality (physics)
• Structural equation modeling
• Rubin causal model
• Mutual Information

External links

• "Transfer Entropy Toolbox". Google Code., a toolbox, developed in C++ and MATLAB, for computation of transfer entropy between spike trains.