The rich get richer (statistics)

From Wikipedia, the free encyclopedia
Jump to: navigation, search

In probability and statistics, the phrase "the rich get richer" is used to describe the self-reinforcing behavior of certain probability distributions and stochastic processes, such as the Dirichlet process and Chinese restaurant process. Note that this behavior is seen in the context of many, perhaps most, distributions, when considering a sequence of independent identically distributed observations drawn from a probability distribution with unknown parameter and examining the conditional distribution of one observation given all previous ones. For example, if an observation is drawn from a Gaussian distribution with unknown mean (with uncertainty expressed by a prior distribution over the mean), then the posterior distribution over the mean will be shifted towards the observation, and the next observation will have a higher probability of being similar to the previous observation than it was under the prior distribution. In this case, regions that are "rich" in the sense of containing a large fraction of the observations are quite likely to get richer, whereas poor regions are less likely to get richer.

However, "the rich get richer" is often used particularly of the Chinese restaurant process (CRP), a stochastic process closely related to the Dirichlet process. In this model, the probability of an observation taking on a specific value is directly proportional to the number of times that value has already been seen. (In the colorful terminology of the CRP, the probability of a customer sitting at a particular infinitely large table in an infinitely large Chinese restaurant is directly proportional to the number of customers already seated at the table.) Note that in order for this model to work tractably, there are also a fixed number of pseudo-observations assumed, and with probability proportional to the number of pseudo-observations,[clarification needed] an observation is given a new value that has never been seen before.

See also[edit]