The fixation index (FST) is a measure of population differentiation due to genetic structure. It is frequently estimated from genetic polymorphism data, such as single-nucleotide polymorphisms (SNP) or microsatellites. Developed as a special case of Wright's F-statistics, it is one of the most commonly used statistics in population genetics.
If is the average frequency of an allele in the total population, is the variance in the frequency of the allele between different subpopulations, weighted by the sizes of the subpopulations, and is the variance of the allelic state in the total population, FST is defined as 
This definition, which is due to Wright, illustrates that FST measures the amount of genetic variance that can be explained by population structure. This can also be thought of as the fraction of total diversity that is not a consequence of the average diversity within subpopulations, where diversity is measured by the probability that two randomly selected alleles are different, namely . If the allele frequency in the th population is and the relative size of the th population is , then
where is the probability of identity by descent of two individuals given that the two individuals are in the same subpopulation, and is the probability that two individuals from the total population are identical by descent. Using this definition, FST can be interpreted as measuring how much closer two individuals from the same subpopulation are, compared to the total population. If the mutation rate is small, this interpretation can be made more explicit by linking the probability of identity by descent to coalescent times: Let T0 and T denote the average time to coalescence for individuals from the same subpopulation and the total population, respectively. Then,
This formulation has the advantage that the expected time to coalescence can easily be estimated from genetic data, which led to the development of various estimators for FST.
In practice, none of the quantities used for the definitions can be easily measured. As a consequence, various estimators have been proposed. A particularly simple estimator applicable to DNA sequence data is:
where and represent the average number of pairwise differences between two individuals sampled from different sub-populations () or from the same sub-population (). The average pairwise difference within a population can be calculated as the sum of the pairwise differences divided by the number of pairs. However, this estimator is biased when sample sizes are small or if they vary between populations. Therefore, more elaborate methods are used to compute FST in practice. Two of the most widely used procedures are the estimator by Weir & Cockerham (1984), or performing an Analysis of molecular variance. A list of implementations is available at the end of this article.
This comparison of genetic variability within and between populations is frequently used in applied population genetics. The values range from 0 to 1. A zero value implies complete panmixis; that is, that the two populations are interbreeding freely. A value of one implies that all genetic variation is explained by the population structure, and that the two populations do not share any genetic diversity.
For idealized models such as Wright's finite island model, FST can be used to estimate migration rates. Under that model, the migration rate is
The interpretation of FST can be difficult when the data analyzed are highly polymorphic. In this case, the probability of identity by descent is very low and FST can have an arbitrarily low upper bound, which might lead to misinterpretation of the data. Also, strictly speaking FST is not a genetic distance, as it does not satisfy the triangle inequality. As a consequence new tools for measuring genetic differentiation continue being developed.
FST in humans
The International HapMap Project estimated FST for three human populations using SNP data. Across the autosomes, FST was estimated to be 0.12. The significance of this FST value in humans is contentious. As an FST of zero indicates no divergence between populations, whereas an FST of one indicates complete isolation of populations, Anthropologists often cite Lewontin's 1972 work which came to a similar value and interpreted this number as meaning there was little biological differences between human races. On the other hand, while an FST value of 0.12 is lower than that found between populations of many other species, Henry Harpending argued that this value implies on a world scale a "kinship between two individuals of the same human population is equivalent to kinship between grandparent and grandchild or between half siblings".
Autosomal genetic distances based on SNPs
|Europe (CEU)||Sub-Saharan Africa (Yoruba)||East-Asia (Japanese)|
|Sub-Saharan Africa (Yoruba)||0.153|
Programs for calculating FST
Modules for calculating FST
- Holsinger, Kent E.; Bruce S. Weir (2009). "Genetics in geographically structured populations: defining, estimating and interpreting FST". Nat Rev Genet 10 (9): 639–650. doi:10.1038/nrg2611. ISSN 1471-0056. PMID 19687804.
- Richard Durrett (12 August 2008). Probability Models for DNA Sequence Evolution. Springer. ISBN 978-0-387-78168-6. Retrieved 25 October 2012.
- Hudson, RR.; Slatkin, M.; Maddison, WP. (Oct 1992). "Estimation of Levels of Gene Flow from DNA Sequence Data". Genetics 132 (2): 583–9. PMC 1205159. PMID 1427045.
- Weir, B. S.; Cockerham, C. Clark (1984). "Estimating F-Statistics for the Analysis of Population Structure". Evolution 38 (6): 1358. doi:10.2307/2408641. ISSN 0014-3820.
- Lewontin, Richard C. (1972). "The apportionment of human diversity". Evolutionary biology 6 (38): 381–398. doi:10.1007/978-1-4684-9063-3_14.
- Harpending, Henry (2002-11-01). "Kinship and Population Subdivision" (PDF). Population & Environment 24 (2): 141–147. doi:10.1023/A:1020815420693. JSTOR 27503827.
- Nelis, Mari et al. (2009-05-08). Fleischer, Robert C., ed. "Genetic Structure of Europeans: A View from the North–East". PLoS ONE 4 (5): e5472. Bibcode:2009PLoSO...4.5472N. doi:10.1371/journal.pone.0005472. PMC 2675054. PMID 19424496., see table
- Tian, Chao et al. (November 2009). "European Population Genetic Substructure: Further Definition of Ancestry Informative Markers for Distinguishing among Diverse European Ethnic Groups". Molecular Medicine 15 (11-12): 371–383. doi:10.2119/molmed.2009.00094. ISSN 1076-1551. PMC 2730349. PMID 19707526., see table
- Crawford, Nicholas G. (2010). "smogd: software for the measurement of genetic diversity". Molecular Ecology Resources 10 (3): 556–557. doi:10.1111/j.1755-0998.2009.02801.x. PMID 21565057.
- Evolution and the Genetics of Populations Volume 2: the Theory of Gene Frequencies, pg 294–295, S. Wright, Univ. of Chicago Press, Chicago, 1969
- A haplotype map of the human genome, The International HapMap Consortium, Nature 2005