- In positive frequency-dependent selection, the fitness of a phenotype increases as it becomes more common.
- In negative frequency-dependent selection, the fitness of a phenotype increases as it becomes rarer. This is an example of balancing selection.
Frequency-dependent selection is usually the result of interactions between species (predation, parasitism, or competition) or between genotypes within species (usually competitive or symbiotic), and has been especially frequently discussed with relation to anti-predator adaptations. Frequency-dependent selection can lead to polymorphic equilibria which result from interactions among genotypes within species, in the same way that multi-species equilibria require interactions between species in competition (e.g. where αij parameters in Lotka-Volterra competition equations are non-zero).
Negative frequency-dependent selection
Perhaps the best known early modern statement of the principle is Bryan Clarke's 1962 paper on apostatic selection (a synonym of negative frequency-dependent selection). Clarke discussed predator attacks on polymorphic British snails, citing Luuk Tinbergen's classic work on searching images as support that predators such as birds tended to specialize on common forms of palatable species. Clarke later argued that frequency-dependent balancing selection could explain molecular polymorphisms (often in the absence of heterosis) in opposition to the neutral theory of molecular evolution.
Another example is plant self-incompatibility alleles. When two plants share the same incompatibility allele, they are unable to mate. Thus, a plant with a new (and therefore, rare) allele has more success at mating, and its allele spreads quickly through the population.
In human pathogens, such as the flu virus, once a particular strain has become common, most individuals have developed an immune response to that strain. But a rare, novel strain of the flu virus is able to spread quickly to almost any individual, causing continual evolution of viral strains.
In behavioral ecology, negative frequency-dependent selection is often responsible for maintaining multiple behavioral strategies within a species. For example, male common side-blotched lizards have three morphs, which either defend large territories and maintain large harems of females, defend smaller territories and keep one female, or mimic females in order to sneak matings from the other two morphs. These three morphs participate in a rock paper scissors sort of interaction such that no one morph completely outcompetes the other two. Another example occurs in the Scaly-breasted Munia, where certain individuals become scroungers and others become producers.
Positive frequency-dependent selection
Where negative frequency-dependent selection gives an advantage to rare phenotypes, positive frequency-dependent selection gives an advantage to common phenotypes. In the between-species analogue, this is equivalent to an Allee effect, in which if a species is too rare, it may decline to extinction. This means that new alleles can have a difficult time invading a population, since they don't experience significant benefit until they become common.
The effect can be seen in the evolution of warning coloration (aposematism) in toxic or distasteful organisms. The signalling theory advantage of such coloration is that predators can learn to avoid potential prey with that color pattern. For example, in the Batesian mimicry complex between a harmless mimic, the scarlet kingsnake (Lampropeltis elapsoides), and the model, the eastern coral snake (Micrurus fulvius), in locations where the model and mimic were in deep sympatry, the phenotype of the scarlet kingsnake was quite variable due to relaxed selection. But where the pattern was rare, the predator population was not 'educated', so the pattern brought no benefit. The scarlet kingsnake was much less variable on the allopatry/sympatry border of the model and mimic, likely due to increased selection since the eastern coral snake is rare, but present, on this border. Therefore the color is only advantageous once it has become common.
- Poulton, E. B. 1884. Notes upon, or suggested by, the colours, markings and protective attitudes of certain lepidopterous larvae and pupae, and of a phytophagous hymenopterous larva. Transactions of the Entomological Society of London 1884: 27–60.
- Allen, J.A., and B.C. Clarke. 1984. Frequency-dependent selection -- homage to Poulton, E.B. Biological Journal of the Linnean Society 23:15-18.
- Clarke, B. 1962. Balanced polymorphism and the diversity of sympatric species. Pp. 47-70 in D. Nichols ed. Taxonomy and Geography. Systematics Association, Oxford.
- Tinbergen, L. 1960. The natural control of insects in pinewoods. I. Factors influencing the intensity of predation in songbirds. Archs.Neerl.Zool. 13:265-343.
- Takahata, N.; Nei, M. (1990). "Allelic genealogy under overdominant and frequency-dependent selection and polymorphism of major histocompatibility complex loci". Genetics 124: 967–78.
- Borghans, JA; Beltman, JB; De Boer, RJ. (Feb 2004). "MHC polymorphism under host-pathogen coevolution.". Immunogenetics 55 (11): 732–9.
- Sinervo, B.; C.M. Lively (1996). "The rock–paper–scissors game and the evolution of alternative male strategies". Nature 380 (6571): 240–243. doi:10.1038/380240a0.
- Sinervo, Barry; Donald B. Miles, W.Anthony Frankino, Matthew Klukowski, Dale F. DeNardo (2000). "Testosterone, Endurance, and Darwinian Fitness: Natural and Sexual Selection on the Physiological Bases of Alternative Male Behaviors in Side-Blotched Lizards". Hormones and Behavior 38 (4): 222–233. doi:10.1006/hbeh.2000.1622. PMID 11104640.
- Barnard, C.J.; Sibly, R.M. "Producers and scroungers: A general model and its application to captive flocks of house sparrows". Animal Behaviour 29 (2): 543–550. doi:10.1016/S0003-3472(81)80117-0.
- Harper, G. R; Pfennig, D. W (22 August 2007). "Mimicry on the edge: why do mimics vary in resemblance to their model in different parts of their geographical range?". Proceedings of the Royal Society B: Biological Sciences 274 (1621): 1955–1961. doi:10.1098/rspb.2007.0558. PMC 2275182. PMID 17567563.
- Tamarin, Robert H. (2001) Principles of Genetics. 7th edition, McGraw-Hill.