# Information gain ratio

Information gain ratio is between 0 and 1, cf. ordinary information gain

## Information Gain Calculation

Let $Attr$ be the set of all attributes and $Ex$ the set of all training examples, $value(x,a)$ with $x\in Ex$ defines the value of a specific example $x$ for attribute $a\in Attr$, $H$ specifies the entropy. The information gain for an attribute $a\in Attr$ is defined as follows:

$IG(Ex,a)=H(Ex) -\sum_{v\in values(a)} \left(\frac{|\{x\in Ex|value(x,a)=v\}|}{|Ex|} \cdot H(\{x\in Ex|value(x,a)=v\})\right)$

The information gain is equal to the total entropy for an attribute if for each of the attribute values a unique classification can be made for the result attribute. In this case the relative entropies subtracted from the total entropy are 0.

## Intrinsic Value Calculation

The intrinsic value for a test is defined as follows:

$IV(Ex,a)= -\sum_{v\in values(a)} \frac{|\{x\in Ex|value(x,a)=v\}|}{|Ex|} * \log_2\left(\frac{|\{x\in Ex|value(x,a)=v\}|}{|Ex|}\right)$

## Information Gain Ratio Calculation

The information gain ratio is just the ratio between the information gain and the intrinsic value: $IGR(Ex,a)=IG / IV$