Evolutionary game theory
Evolutionary game theory (EGT) is the application of game theory to evolving populations of lifeforms in biology. EGT is useful in this context by defining a framework of contests, strategies, and analytics into which Darwinian competition can be modelled. EGT originated in 1973 with John Maynard Smith and George R. Price's formalisation of the way in which such contests can be analysed as "strategies" and the mathematical criteria that can be used to predict the resulting prevalence of such competing strategies.
Evolutionary game theory differs from classical game theory by focusing more on the dynamics of strategy change as influenced not solely by the quality of the various competing strategies, but by the effect of the frequency with which those various competing strategies are found in the population.
Evolutionary game theory has proven itself to be invaluable in helping to explain many complex and challenging aspects of biology. It has been particularly helpful in establishing the basis of altruistic behaviours within the context of Darwinian process. Despite its origin and original purpose, evolutionary game theory has become of increasing interest to economists, sociologists, anthropologists, and philosophers.
- 1 The problem
- 2 Classical game theory
- 3 Adapting game theory to evolutionary games
- 4 Models
- 5 Games
- 6 Hawk Dove
- 7 The War of Attrition game
- 8 Strategic alternatives in social behaviour
- 9 Who is playing the game?
- 10 Prisoners dilemma
- 11 Routes to altruism
- 12 The ESS
- 13 Rock Scissors Paper game
- 14 Signalling, sexual selection and the handicap principle
- 15 Co-evolution
- 16 Extensions of the evolutionary game theory model
- 17 Further application
- 18 See also
- 19 References
- 20 Sources
- 21 Further reading
- 22 External links
"An ounce of algebra is worth a ton of verbal argument" J.B.S. Haldane (as quoted by John Maynard Smith)
The need for evolutionary game theory in biology started with a problem. The problem was how to explain ritualized animal behaviour in a conflict situation; "why are animals so 'gentlemanly or ladylike' in contests for resources?" This was a problem that leading ethologists Niko Tinbergen and Konrad Lorenz were trying to address. Tinbergen proposed that such behaviour exists for the benefit of the species. Maynard Smith couldn't see this reasoning matched with Darwinian thought as he understood it, where selection occurs at an individual level and therefore defections to self-interest are rewarded while seeking the common good is not. Maynard Smith, a former engineer and highly competent mathematician, turned to game theory to answer the question. The suggestion that game theory might solve the problem was made by George Price even though some unsuccessfully applied efforts by Richard Lewontin at using game theory had failed. Maynard Smith’s way of applying this mathematics proved to be a far more realistic model, and came to be a major mathematical contribution in the field of theoretical biology.
Classical game theory
Game theory was conceived by John von Neumann to mathematically determine optimal strategies in situations in which a competition between adversaries is taking place. A contest involves a number of players, all of whom have a choice of moves for the game. Games can be a single round or repetitive. The approach a player has in making his or her moves constitutes the player's "strategy". Rules govern the outcome for the set of moves taken by the players and outcomes produce payoffs for the various players; rules and resulting payoffs can be expressed as decision trees or in a payoff matrix.
Classical game theory essentially requires that all of the players make rational choices (that is, making their strategic choice on a wholly rationally-determined evaluation of probable outcomes). As a consequence, it is fundamental in game theory that each player must consider the strategic analysis that the players' opponents are making in determining that his or her own strategic choice is appropriate.
Adapting game theory to evolutionary games
On reflection Maynard Smith realised that in an evolutionary version of Game Theory it is not really a requirement that players be rational – it is only required that they have a strategy. The results of the game will test how good that strategy is. That is what Evolution does – it tests alternative strategies for the ability to survive and reproduce. In Biology, strategies are genetically inherited traits that control an individual's action – and strategies are algorithmic – just like computer programs. The key point in the evolutionary game theory model is that the success of a strategy is not just determined by how good the strategy is in itself, it is a question of how good the strategy is in the presence of other alternative strategies, and of the frequency that other strategies are employed within a competing population. It is also a question of how good a strategy plays against itself, because in the biological world a successful strategy will eventually dominate a population and competing individuals in it end up facing identical strategies to their own.
The object of the evolutionary game is to become more fit than competitors – to produce as many replicas of oneself as one can and the payoff is in units of Fitness (relative worth in being able to reproduce). It is always a multi-player game with a very large population of competitors. Rules describe the contest as in classical Game Theory but for evolutionary games rules include the element of replicator dynamics, in other words the general rules say exactly how the fitter players will spawn more replicas of themselves into the population and the less fit will be culled out of the player population (expressed in a Replicator Equation). The replicator dynamics in essence models the heredity mechanism, but for simplicity leaves out mutation. Similarly, Evolutionary Game Theory only uses asexual reproduction for the sake of simplicity. Games are run repetitively with no terminating conditions. The results that are studied include the dynamics of changes in the population, the success/survival of strategies and any equilibrium states reached in a competing. Unlike classical Game Theory players do not choose their strategy or have the ability to change it, they are born with that strategy and their offspring will inherit that same identical strategy.
It is important to realize that EGT is not just a specialist mathematical treatment of animal contests to determine their dynamics and results, but in a manner similar to the field of Evolutionary Algorithms, it is a comprehensive model that encompasses the Darwinian process itself – including the central tenets of competition (the game) natural selection (replicator dynamics) and heredity - all within the overall model. Therefore it is a major vehicle to help understand and explain some of the most fundamental questions in biology including the issue of group selection, sexual selection, altruism, parental care, co evolution, and ecological dynamics. Much of the progress in developing understanding in these diverse areas has been aided by evolutionary game theory modelling and many of the counter intuitive situations in these areas have been explained and put on a firm mathematical footing by the use of these models.
The common methodology to study the evolutionary dynamics in games is through replicator equations. These replicator equations in the context of evolutionary biology shows the growth rate of the proportion of organisms using a certain strategy and that rate is equal to the difference between the average payoff of that strategy and the average payoff of the population as a whole. Continuous replicator equations assume infinite populations, continuous time, complete mixing and that strategies breed true. The attractors (stable fixed points) of the equations are equivalent with evolutionarily stable states. A strategy which can survive all "mutant" strategies is considered evolutionary stable. In the context of animal behavior, this usually means such strategies are programmed and heavily influenced by genetics, thus making any player or organism's strategy determined by these biological factors.
In Game Theory various games are mathematical objects of different character — in rules, relative payoffs, and attendant mathematical behaviours. Each "game" represents different situations — in the kinds of problems that organisms have to deal with, and the possible strategies that they might adopt if they are to successfully survive and reproduce. To achieve a better feel for the challenges of these different situations, evolutionary games are often given rather colourful names and “cover stories” which quite effectively describe the general situation in which the particular game places its players. It all helps develop a feel for the mathematics of the game and the problems the players face.
Some representative games of evolutionary game theory are hawk-dove, war of attrition, stag hunt, producer-scrounger, tragedy of the commons, and prisoner's dilemma. Some of the various strategies that apply in these games are Hawk, Dove, Bourgeois, Prober, Defector, Assessor, and Retaliator. Depending on the particular “Game” the various strategies vie against one another under the particular game rules, and the mathematics of the evolutionary game theory are used to determine the results and behaviours.
The most classic game (and Maynard Smith's starting point) is the Hawk Dove game. The game was conceived to analyse the animal contest problem highlighted by Lorenz and Tinbergen. It is a contest over a sharable resource, say some food. The contestants can be either a Hawk or a Dove. These are not two separate species of bird; they are two subtypes of one species with two different types of strategy (two different morphs). The term Hawk Dove was coined by Maynard Smith because he did his work during the Vietnam War when political views fell into one of these two camps. The strategy of the Hawk (a fighter strategy) is to first display aggression, then escalate into a fight till he either wins or is injured. The strategy of the Dove (fight avoider) is to first display aggression but if faced with major escalation by an opponent to run for safety. If not faced with this level of escalation the Dove will attempt to share the resource.
|meets Hawk||meets Dove|
|if Hawk||V/2 - C/2||V|
Given that the resource is given the value V, the damage from losing a fight is given cost C:
- If a Hawk meets a Dove he gets the full resource V to himself
- If a Hawk meets a Hawk – half the time he wins, half the time he loses…so his average outcome is then V/2 minus C/2
- If a Dove meets a Hawk he will back off and get nothing - 0
- If a Dove meets a Dove both share the resource and get V/2
The actual payoff however depends on the probability of meeting a Hawk or Dove, which in turn is a representation of the percentage of Hawks and Doves in the population when a particular contest takes place. But that population makeup in turn is determined by the results of all of the previous contests before the present contest- it is a continuous iterative process where the resultant population of the previous contest becomes the input population to the next contest. If the cost of losing C is greater than the value of winning V (the normal situation in the natural world) the mathematics ends in an ESS – an evolutionarily stable strategy situation having a mix of the two strategies where the population of Hawks is V/C. The population will progress back to this equilibrium point if any new Hawks or Doves make a temporary perturbation in the population. The solution of the Hawk Dove Game explains why most animal contests involve only “ritual fighting behaviours” in contests rather than outright battles. The result does not at all depend on “good of the species” behaviours as suggested by Lorenz, but solely on the implication of actions of “selfish genes”.
The War of Attrition game
In the Hawk Dove game the resource is sharable, which gives payoffs to both Doves meeting in a pairwise contest. In the case where the resource is not sharable but an alternative resource might be available by backing off and trying elsewhere, pure Hawk or Dove strategies become less effective. If an unshareable resource is combined with a high cost of losing a contest (injury or possible death) both Hawk and Dove payoffs are then further diminished. A safer strategy of lower cost display, bluffing and waiting to win, then becomes viable – a Bluffer strategy. The game then becomes one of accumulating costs, either the costs of displaying or costs of prolonged unresolved engagement. It’s effectively an auction; the winner is the contestant who will swallow the greater cost while the loser gets, for all his pains, the same cost as the winner but NO resource. The resulting evolutionary game theory mathematics leads to an optimal strategy of timed bluffing.
This is because in the war of attrition any strategy that is unwavering and predictable is unstable – unstable in the sense that such a strategy will ultimately be displaced by a mutant strategy which will simply rely on the fact that it can best the existing predictable strategy just by investing just an extra small delta of waiting resource to insure that it wins. Therefore only a random unpredictable strategy can maintain itself in a population of Bluffers. The contestants in effect choose an “acceptable cost” to be incurred related to the value of the resource being sought – randomly selected at contest start - effectively making a related random “bid” which becomes part of a mixed strategy (a strategy where a contestant has several, or even many, possible actions in his strategy). This implements a distribution of bids for a resource of specific value V, where the particular bid made for any specific contest is chosen at random from within that distribution. The distribution (an ESS) can be computed by invoking the Bishop-Cannings theorem, which holds true for any case of mixed strategy ESS. The distribution function in these contests was thus determined by Parker and Thompson to be:
This leads to the result where the cumulative population of quitters of the contest versus for any particular cost m in this "mixed strategy" solution is:
which is shown in the adjacent graph. The intuitive sense that greater values of resource sought leads to greater waiting times is borne out here.
This is exactly what is observed in nature for contests in a number of species, for example between male dung flies contesting for mating sites. The timing of disengagement in these contests follows the exact mathematical curve derived from the evolutionary theory mathematics.
Asymmetries that allow new strategies
In the War of Attrition there must be nothing that signals the size a bid to an opponent, otherwise this can act as a cue which can be utilised by an opponent for an effective counter-strategy for a mutant. There is however a mutant strategy which can better that of Bluffer in the War of Attrition Game if such an asymmetry exists. This mutant strategy is the Bourgeois strategy (Maynard Smith named the strategy Bourgeois, because with his background of communism he regarded it as “politically bourgeois” way to value ownership). Bourgeois uses an asymmetry of some sort to break the deadlock. In nature one such asymmetry is possession - which contestant has the prior possession of the resource. The strategy is to play a Hawk if in possession of the resource but to display then retreat if not in possession. This strategy requires greater cognitive capability than Hawk but Bourgeois is still a very prevalent strategy in many animal contests… some examples being Mantis Shrimp and Speckled Wood Butterfly contests. Thus, evolutionary game theory explains a very wide range of rather mystifying behaviours in animals and in plant contests for many species with an illuminating clarity.
Games like Hawk Dove and War of Attrition represent pure competition between individuals and have no attendant social element to the game. Where social influences apply there are four possible alternatives for strategic interaction that exist for the competitors. This is shown on the adjacent figure, where a plus sign represents a benefit and a minus sign represents a cost.
- In a Cooperative or Mutualistic relationship both “donor” and “recipient” are almost indistinguishable as both gain a benefit in the game by co-operating, i.e. the pair are in a game-wise situation where both can gain by executing a certain strategy, or alternatively both must act in concert because of some encompassing constraints that effectively puts them “in the same boat”.
- In an Altruistic relationship the donor, at a cost to himself provides a benefit to the recipient. In the general case the recipient will have a kin relationship to the donor and the donation is one-way. Behaviours where benefits are donated alternatively (in both directions) at a cost, are often called altruistic, but on analysis such “altruism” can be seen to arise from optimised “selfish” strategies
- Spite is essentially a “reversed” form of altruism where an ally is aided by damaging the ally’s competitor(s). The general case is that the ally is kin related and the benefit is an easier competitive environment for the ally. Note: George Price, one of the early mathematical modellers of both altruism and spite, found this equivalence particularly disturbing at an emotional level.
- Selfishness is the base criteria of all strategic choice from a game theory perspective – strategies not aimed at self-survival and self-replication are not long for any game. Critically however, this situation is impacted by the fact that competition is taking place on multiple levels - i.e. at a genetic, an individual and a group level.
The rationale and the mathematics that lie behind adopting one of these alternative social strategies will be covered in the following sections of this article.
Who is playing the game?
At first glance it may be surmised that the contestants of evolutionary games are the individuals present in each generation who directly participate in the game. On reflection however we see that individuals live only through one game cycle, and instead it is the strategies that really contest with one another over the duration of these recursive games. So it is ultimately genes that play out a full contest – genes of STRATEGY (i.e. selfish genes). The contesting genes are not just present in an individual and his/her direct linage; they are also present to a relational degree in all of the individual’s kin. This can sometime profoundly affect the kinds of strategies that will survive, and nowhere is this more pertinent than in issues of cooperation and defection. William Hamilton, whose work defined kin selection, was involved in much of the groundbreaking mathematical work in this area and Hamilton treated many of these cases using game theoretic models. Kin related treatment of game contests help to explain many aspects of the behaviours of eusocial insects, the altruistic behaviour in parent/offspring interactions, mutual protection behaviours, and co-operative care of offspring. For such games Hamilton defined an extended form of fitness which is operative – Inclusive fitness, which extends the fitness measure to include an individual’s offspring as well as any other “offspring equivalents” found in kin.
|The Mathematics of Kin Selection|
|The concept of Kin Selection can be formulated into a mathematical relationship by expressing the basic definition that:
It is worth pointing out that fitness is measured relative to the average population at large, e.g. a fitness=1 means growth at the same average rate of the population, fitness<1 means decreasing percentage-wise in the population (dying out), fitness>1 means increasing percentage-wise in the population (taking over).
The inclusive fitness of an individual wi is the sum of its specific fitness of itself ai plus the specific fitness of each and every relative weighted by the degree of relatedness which equates to the summation of all rj*bj....... where rj is relatedness of a specific relative and bj is that specific relatives fitness - producing:
Now if individual ai sacrifices his "own average equivalent fitness of 1" by accepting a fitness cost C, and then to "get that loss back", wi must still be 1 (or greater than 1)...AND if we use R*B to represent the summation we get:
Hamilton then went on beyond areas of kin relatedness to work with Robert Axelrod to analyze games that involved co-operation under conditions not involving kin where reciprocal altruism comes into play.
“Ants are good citizens, they place group interests first” Clarence Day
Insect societies have always been a source of fascination and inscrutability. Certainly one of the very most inexplicable behaviours of these eusocial insects is the forfeiture of reproductive rights that the workers grant in favour of the queen. In a Darwinian sense, no greater sacrifice can ever exist. The explanation that has been forthcoming is in Kin Selection influences that arise from the genetic makeup of these workers which predisposes them to such kin oriented altruistic behaviours . Most eusocial insect societies have haplo-diplod sexual determination, which in essence means that males develop from unfertilised eggs, females from fertilised. This leads to the situation in these Haplodiploid species, that sisters share 75% of their genes in common…. in effect more than they genetically share with their mother. But in this system ALL the other workers are sisters, further skewing the kin related payoffs toward altruistic sacrifice. Helping to raise the queen's offspring, which are also sisters, therefore has major payoffs as does sacrificing onself if the nest is invaded by some predator. For those insect societies which do not exhibit haplodiplodal genetic relationships, the most significant example being termites, there still exist kin related influences arising from inbreeding, monogamy and the ineffectiveness of dispersal 
The widely accepted kin related explanation of insect eusociality has however been challenged recently by a few highly noted evolutionary game theorists (Nowak and Wilson)  who have published a controversial alternative game theoretic explanation based on a sequential development and group selection effects proposed for these insect species.
One of the great difficulties of Darwinian Theory, and one recognised by Darwin himself was the problem of altruism. If the basis for selection is at the individual level, altruism makes no sense at all. But universal selection at the group level (for the good of the species, not the individual) fails to pass the test of the mathematics of game theory and is certainly not found to be the general case in nature. Yet in many social animals altruistic behaviour can be found. The solution to this paradox can be found in the application of evolutionary game theory to the “Prisoners Dilemma” game - a game which tests the payoffs of cooperating or in defecting from cooperation. It is certainly the most studied game in all of Game Theory.
As with all games in evolutionary game theory the analysis of Prisoners Dilemma is as a repetitive game. This repetitive nature affords competitors the possibility of retaliating for “bad behaviour” (defection) in previous rounds of the game. There is a multitude of strategies which have been tested by the mathematics of EGT and in computer simulations of contests and the conclusion is that the best competitive strategies are general cooperation with a reserved retaliatory response if necessary. The most famous and certainly one of the most successful of these strategies is Tit for Tat which carries out this approach by executing the simplest of algorithms.
procedure tit-for-tat EventBit=Trust; do while Contest=ON; if Eventbit=Trust then Cooperate else Defect; if Opponent_Move=Cooperate then EventBit=Trust else Eventbit=NOT(Trust); end;
The pay-off for any single round of the game is defined by the pay-off matrix for a single round game (shown in bar chart 1 below). In multi-round games the different choices - Co-operate or Defect - can be made in any particular round, resulting in a certain round payoff. It is, however, the possible accumulated pay-offs over the multiple rounds that count in shaping the overall pay-offs for differing multi-round strategies such as Tit-for-Tat.
Example 1: This is the straightforward single round Prisoners Dilemma Game (the game is executed in only one round). The classic Prisoners Dilemma game payoffs gives you a maximum payoff if you defect and your "partner" co-operates (this choice for you is known as TEMPTATION). If however you co-operate and your partner defects you get the worst possible result ( the SUCKERS PAYOFF ). In these payoff conditions the best choice (a Nash Equilibrium in classic game theory) is to defect.
Example 2: This is Prisoners Dilemma played repeatedly. The strategy employed is Tit-for-Tat which alters behaviors based on the action taken by a partner in the previous round - i.e. reward co-operation and punish defection. The effect of this strategy in ACCUMULATED payoff over many rounds is to produce a higher payoff for both players co-operation and a lower payoff for defection. This removes the Temptation to defect. The suckers payoff also becomes less, although "invasion" by a pure defection strategy is not entirely eliminated (this can be overcome by clustering of cooperative strategies - see spatial games below).
Routes to altruism
Altruism takes place when one individual, at a cost C to itself, exercises a strategy that provides a benefit B to another individual. The cost may consist of a loss of capability or resource which helps in the “battle for survival and reproduction” or it may consist of an added risk to its own survival – i.e. incurring a higher probability of non-survival for itself. Altruism strategies arise in a number of ways:
|Type||Applies to:||Situation||Mathematical effect|
|Kin Selection – (inclusive fitness of related contestants)||Kin – genetically related individuals||Evolutionary Game participants are actually “genes of strategy”. So it follows that what is the best payoff for an individual is not necessarily what is the best payoff for the gene. In any generation the player gene is NOT only in one individual, it is in a Kin-Group. What produces the highest fitness payoff for the Kin Group is the better strategy and will be selected by natural selection. Therefore strategies that include “self-sacrifice” on the part of individuals are often the “game winner” – the evolutionarily stable strategy. Of course animals must necessarily live in kin-group during some life phase or over all of the game to afford the opportunity for this altruistic sacrifice to ever take place.||Games must take into account Inclusive Fitness. Fitness function is the combined fitness of a group of related contestants – each weighted by the degree of relatedness – relative to the total genetic population. The mathematical analysis of this gene centric view of the game leads to "Hamilton's rule" that the genetic relatedness of the altruistic donor must exceed the cost-benefit ratio of the altruistic act itself, in mathematical terms:
|Direct reciprocity||Related or non related contestants that “trade favours” in paired relationships||A game theoretic embodiment of the adage “I’ll scratch your back if you scratch mine”. A pair of individuals exchange favours in a multi-round game in which both individuals are involved. The individuals in the pair are specifically recognisable to one another as partnered. The term “DIRECT” applies because the return favour is specifically given back to the pair partner only.||The characteristics of the multi-round game produce a danger of “defection” and the potentially lesser payoffs of “cooperation” in each round, however any such defection can lead to subsequent "punishment" in a following round – establishing the game as that of classical repeated Prisoners Dilemma. Therefore, the family of tit-for-tat strategies come to the fore.|
|Indirect Reciprocity||Related or non related contestants “trade favours” but not with partnering involved. A return favour is “implied” but with no specific identified source who is to give it.||This behaviour is akin to “I’ll scratch your back, you scratch someone elses back, another someone else will scratch mine (probably)". The return favour is not derived from any particular established partner. The potential for indirect reciprocity exists for a specific organism if it lives in a cluster of individuals who can interact over an extended period of time.
It has been argued that human behaviours in establishing moral system as well as the expending of significant energies in human society for tracking individual reputation is a direct effect of societies reliance on strategies of indirect reciprocation.
|The game is highly susceptible to defection, particularly as direct retaliation becomes essentially impossible. Therefore indirect reciprocity will not work mathematically unless some additional factor is at play to verify that any individual being granted a favour is “trustworthy” to return the favour in the sharing group and insuring the punishment (non-reward) of defectors for “misbehaving”. This requires essentially keeping a "social score", a measure of the past co-operative behaviour. The mathematics leads to a modified version of Hamilton's Rule where:
Organisms that use this sort of social score as an extended modification of the tit-for-tat variant strategies are termed Discriminators. Different discrimination algorithms exist for determining social scores, but in general all of these require a much higher level of cognition to exist than strategies of simple direct reciprocity. As David Haig, a noted evolutionary biologist succinctly put it - "For direct reciprocity you need a face; for indirect reciprocity you need a name".
The Evolutionarily Stable Strategy (ESS) is perhaps the most widely known albeit most widely misunderstood concept in evolutionary game theory. The ESS is basically akin to Nash Equilibrium in classical Game Theory, but with mathematically extended criteria.
Nash Equilibrium is a game equilibrium where it is not rational for any player to deviate from the present strategy they are executing. As discussed, in Evolutionary game Theory contestants are NOT behaving with rational choice, nor do they have the ability to totally alter their strategy, aside from executing a very limited “mixed strategy”. An ESS is instead a state of game dynamics where, in a very large (or infinite) population of competitors, another mutant strategy cannot successfully enter the population to disturb the existing dynamic (which in itself is population mix dependent). This leads to a situation where to be a successful strategy having an ESS, the strategy must be both effective against competitors when it is rare - to enter the previous competing population, and also successful when later in high proportion in the population - to “defend itself”. This in turn necessarily means that the strategy needs to be successful when it contends with others exactly like itself.
ESS is NOT:
- An OPTIMAL strategy – an optimal strategy would maximize Fitness, and many ESS states are far below the maximum fitness achievable in a fitness landscape. (see Hawk Dove graph above as an example of this)
- A singular solution – often several ESS conditions can exist in a competitive situation. A particular contest might stabilize into any one of these possibilities, but later a major perturbation in conditions can move the solution into one of the alternative ESS states.
- Always present - It is also possible for there to be no ESS. An example evolutionary game with no ESS is the Rock-Scissors-Paper game found in a number of species (an example the side-blotched lizard (Uta stansburiana))
- An unbeatable strategy - The ESS strategy is not necessarily an unbeatable strategy, it is only an uninvadable one.
The ESS state can be solved for mathematically by exploring either the dynamics of population change to determine any ESS.... or alternatively by solving equations for the stable stationary point conditions which fundamentally define an ESS. For example, in the Hawk Dove Game we can look for whether there is a static population mix condition where the fitness of Doves will be exactly the same as fitness of Hawks (therefore both having equivalent growth rates - a "static point").
Let chance of meeting a Hawk=p so therefore the chance of meeting a dove is (1-p)
Let WHawk equal the Payoff for Hawk.....
WHawk=Payoff in the chance of meeting a Dove + Payoff in the chance of meeting a Hawk
Taking the PAYOFF MATRIX results and plugging them into the above equation:
Similarly for a Dove:
Equating the two fitnesses, Hawk and Dove
... and solving for p
so for this "static point" where the Population Percent is an ESS solves to be ESS(percent Hawk)=V/C
Similarly using inequalities it can be shown that an additional Hawk or Dove “mutant” entering this ESS state generates a situation leading eventually to LESS fitness for their kind – both a true Nash and an ESS equilibrium. This fairly simple example shows that when the risks of contest injury or death (the Cost C) is significantly greater than the potential reward offered (the benefit value V) then the stable population which is reached will be MIXED between the aggressors and the doves, and that the proportion of doves will exceed that of the aggressors. This then mathematically explains behaviours that are actually observed in nature.
Rock Scissors Paper game
An evolutionary game that actually turns out to be a children’s game is rock-paper-scissors. The game is simple – rock bests scissors (blunts it), scissors bests paper (cuts it), and paper bests rock (wraps it up). Anyone who has ever played this simple game knows that it is not sensible to have any favoured play – your opponent will soon notice this and switch to the winning counter-play. The best strategy (a Nash equilibrium) is to play a mixed random game with any of the three plays taken a third of the time. This, in EGT terms, is a mixed strategy. But many lifeforms are incapable of mixed behavior — they only exhibit one strategy (known as a “pure” strategy”). If the game is played only with the pure Rock, Scissor and Paper strategies the evolutionary game is dynamically unstable: Rock mutants can enter an all scissor population, but then – Paper mutants can take over an all Rock population, but then – Scissor mutants can take over an all Paper population – and on and on…. This is easily seen on the game payoff matrix, where if the paths of mutant invasion are noted, it can be seen that the mutant "invasion paths" form into a loop. This in triggers a cyclic invasion behaviour.
The Side-Blotched Lizard
The Side-Blotched Lizard (Uta stansburiana) is polymorphic with three morphs that each pursues a different mating strategy
- 1) The Orange throat is very aggressive and operates over a large territory - attempting to mate with numerous females within this larger area
- 2) The unaggressive Yellow Throat (called “sneakers”) mimic the markings/behavior of female lizards and sneakily slip into the Orange Throat's territory to mate with the females there (thereby overtaking the population), and
- 3) The Blue Throat who mates with and carefully guards ONE female - making it impossible for the sneakers to succeed and therefore overtakes their place in a population…
However the BlueThroats cannot overcome the more aggressive Orange Throats… And on, and on… The overall situation corresponds to the exact form of the Rock, Scissors, Paper game and the dynamics is much the same. The populations for these lizards actually cycle on a six year basis. Once again EGT explains a very curious and otherwise inexplicable behaviour in the field of biology. When he read that lizards of the species Uta stansburia were essentially engaged in a game with rock-paper-scissors structure John Maynard Smith exclaimed `They have read my book!´
RPS and ecology
Rock Paper Scissors, incorporated into an evolutionary game has been used for modeling natural processes in the study of Ecology. In two-strategy “pairwise” contests, usually one strategy becomes dominant then maintains its dominance, bringing the system to a static ESS point. Interpreted as modelling the competition between species in nature, this suggests that one species will tend to dominate a population of organisms occupying a single ecological niche. However, most ecologies support many species, each dominating its own niche. With more than two species present, mechanisms come into play which bring more complex dynamics to these systems. For example, competition occurring now between two of the species might indirectly help a third by weakening a future rival before direct competition takes place. In such cases, the ESS point can take on a more complex character; even developing a distribution of species which is cyclic over time. The Rock Paper Scissors game was used as the basis for modelling these multi-species ecologies, producing dynamics closely matching observations from nature.
Signalling, sexual selection and the handicap principle
Aside from the difficulty of explaining how altruism exists in many evolved organisms Darwin was also bothered by a second conundrum –why do a significant number of species have phenotypical attributes that are patently disadvantageous to them with respect to their survival - and should by the process of natural section be selected against – e.g. the massive inconvenient feather structure found in a peacock's tail? Regarding this issue Darwin wrote to a colleague “ The sight of a feather in a peacocks tail, whenever I gaze at it, makes me sick”. It is the mathematics of evolutionary game theory, which has not only explained the existence of altruism but also explains the totally counterintuitive existence of the peacock’s tail and other such biological encumbrances.
On analysis, problems of biological life are not at all unlike the problems that define economics – eating (akin to resource acquisition and management), survival (competitive strategy) and reproduction (investment, risk and return). Game theory was originally conceived as a mathematical analysis of economic processes and indeed this is why it has proven so useful in explaining so many biological behaviours. One important further refinement of the EGT model that has particular economic overtones rests on the analysis of COSTS. A simple model of cost assumes that all competitors suffer the same penalty imposed by the Game costs, but this is not the case. More successful players will be endowed with or will have accumulated a higher “wealth reserve” or “affordability” than less successful players. This wealth effect in evolutionary game theory is represented mathematically by “resource holding potential (RHP)” and shows that the effective cost to a competitor with higher RHP are not as great as for a competitor with a lower RHP. As a higher RHP individual is more desirable mate in producing potentially successful offspring, it is only logical that with sexual selection RHP should have evolved to be signalled in some way by the competing rivals, and for this to work this signalling must be done honestly. Amotz Zahavi has developed this thinking in what is known as the Handicap Principle, where superior competitors signal their superiority by a costly display. As higher RHP individuals can properly afford such a costly display this signalling is inherently honest, and can be taken as such by the signal receiver. Nowhere in nature is this better illustrated than in the magnificent and costly plumage of the male Peacock. The mathematical proof of the handicap principle was developed by Alan Grafen using evolutionary game-theoretic modelling.
Two types of dynamics have been discussed so far in this article:
- Evolutionary games which lead to a stable situation or point of stasis for contending strategies which result in an evolutionarily stable strategy
- Evolutionary games which exhibit a cyclic behaviour (as with RPS game) where the proportions of contending strategies continuously cycle over time within the overall population
A third dynamic can exist in certain more complex systems, where combinations of effects are in play, which contain not only intra-species competition but also inter-species competition as well. This is the realm of co-evolution. Examples include predator-prey competition and host-parasite competition. There are also co-evolutionary situations possible where two species interact where mutual interests are at play in the dynamic. Evolutionary game models have been created to encompass both these co-evolutionary situations for pairwise and multi-species coevolutionary systems. The general dynamic differs between competitive systems and mutualistic systems.
In competitive (non-mutualistic) inter-species coevolutionary system the species are involved in an “arms race” - where adaptions that are better at competing against the other species are less likely to be culled. Both game payoffs and replicator dynamics are thereby affected to reflect this. The “counterstrategy” of the other competitor species is similarly affected, exposing them to a positive selection pressure for any effective counter-strategy – in effect creating an overall competitive arms race. Underlying this total situation is an ecological effect in which the population dynamic of the competing species itself influences outcomes- e.g. a mutant strategy in a prey population which enables that species to drive its sole prey species to extinction is itself doomed. This all leads in effect to a “Red Queen” dynamic where, as in Alice in Wonderland, the protagonists must “run as fast as they can to just stay in one place”.
A number of EGT models have been produced to encompass coevolutionary situations. Modelling these multi-component coevolutionary systems mathematically is necessarily complex. This is particularly true, as a key factor applicable in these coevolutionary systems is the continuous adaption of strategy taking place in the dynamics of such arms races. Most other evolutionary games, e.g. resource conflicts, do not need to include this process of dynamic mutation in the analysis of game dynamics itself. Therefore coevolutionary modelling often involves a cross-over area, where both EGT and Genetic Algorithms reflecting mutational effects are used in the models, often having computers simulate the dynamics of the overall coevolutionary game. The resulting dynamics are studied as various parameters are modified. Because a multitude of variables are simultaneously at play in these complex dynamics, solutions become the province of muli-variable optimality, rather than simply a single ESS. The mathematical criteria of determining stable points are Pareto efficiency and Pareto dominance, which is a measure of solution optimality peaks in these interdependent multivariable systems.
In a paper by Carl Bergstrom and Michael Lachmann, the authors successfully apply evolutionary game theory models to understand the division of benefits in mutualistic interactions between organisms. Darwinian assumptions about fitness are modeled using replicator dynamics to show that the organism evolving at a slower rate in a mutualistic relationship will gain a disproportionately high share of the benefits or payoffs. This application of EGT provided an interesting and perhaps unexpected twist on the Red Queen Hypothesis which concludes evolution favored faster rates of evolution.
Extensions of the evolutionary game theory model
A mathematical model analysing the behaviour of a system needs initially to be as simple as possible to aid in developing a base understanding the fundamentals, or “first order effects”, pertaining to what is being studied. With this understanding in place it is then appropriate to see if other, more subtle, parameters (second order effects) further impact the primary behaviours or shape additional behaviours in the system. Following Maynard Smith’s seminal work in EGT, the subject has had a number of very significant extensions which have shed more light on understanding evolutionary dynamics, particularly in the area of altruistic behaviors. Some of these key extensions to EGC are:
|Spatial Games||There are several very important influences attributable to geographic factors in the field of evolution – not least of which is gene flow and horizontal gene transfer. Spatial game models take geometry into effect by locating contestants in a lattice of cells over a two dimensional plane to represent this geographic element. In spatial games contests take place only with immediate neighbours. Winning strategies take over these immediate neighbourhoods and then go on to interact further with adjacent neighbourhoods. This model has been especially useful in showing how pockets of co-operators can invade and introduce altruism in the Prisoners Dilemma game, where for example Tit for Tat (TFT) is a Nash Equilibrium but NOT also an ESS. TFT cannot normally invade an all Defector population yet in spatial games this becomes possible.|
|Effects of having information||In conventional Game Theory the effect of Signalling (the acquisition of information) is of critical importance in outcomes. This can also be the case in EGT. Nowhere is this more important than in the area of studying Indirect Reciprocity in Prisoners Dilemma (where contests between the SAME paired individuals is NOT repetitive). This models the reality of most normal social interactions which are non-kin related. It has been shown that unless a probability measure of REPUTATION is available in Prisoners Dilemma only DIRECT reciprocity can be achievable. With this information INDIRECT reciprocity is also supported.
Alternatively, agents might have access to an arbitrary signal that is initially uncorrelated to strategy but becomes correlated due to evolutionary dynamics. This is known as the green-beard effect or evolution of ethnocentrism in humans. Depending on the game, It can allow the evolution of either cooperation or irrational hostility.
More generally, there are many instances in biology from the molecular to the multicellular levels where a signaling game model, with information asymmetry between sender and receiver, might be appropriate, such as in mate attraction  or evolution of translation machinery from RNA strings.
|Finite populations||Many evolutionary games have been modelled in finite populations rather than infinite to see the effect this may have. Though in many cases this does not significantly alter game dynamics in others significant differences can and do occur, for example being in the prevalence of mixed strategies.|
The successful application of game theory to evolution has brought further insights to human behavior. Whereas game theory traditionally assumes rational actors, in the real world this does not always describe human behavior. EGT has predicted behaviors in animals where strong assumptions of rationality cannot be made. An important feature of all these sets of models under the umbrella of evolutionary game theory is repetition. If the games were not repeated, these EGT models would not be able to provide any insight into adaptive behaviors and strategies due to the dynamic nature of the mechanisms of evolution. Further, this biological application is meaningful for economics because it provides an understanding of the adjustments that occur between two equilibria (Samuelson 2002). While game theory provides a framework within which biologists can learn and understand organisms, the observation of evolution and how these strategies are applied helps economics illuminate processes. It is interesting to note however, that Maynard Smith himself never felt that the use of evolutionary game theory in the field of economics was quite relevant. In a new introduction for his book "The Alchemy of Finance", George Soros says: "While I still cannot claim full familiarity with the literature, I see some merit in behavioral economics and even more merit in evolutionary game theory. I consider the latter a practical exploration of reflexivity."
- Adaptive dynamics
- Behavioral ecology
- Dynamical systems
- Evolution and the Theory of Games (book)
- Evolutionarily stable strategy
- Gene-centered view of evolution
- Evolutionary dynamics
- Maynard-Smith, J.; Price, G. R. (1973). "The Logic of Animal Conflict". Nature 246 (5427): 15. Bibcode:1973Natur.246...15S. doi:10.1038/246015a0.
- Cohen, Marek, "A Reason for Everything",2004,Faber and Faber,p.231-240, ISBN 0-571-22393-1
- Video Interview - John Maynard Smith - The creation of Evolutionary Game Theory
- Vincent, Thomas, "Evolutionary Game Theory, Natural Selection, and Darwinian Dynamics" 2005, Cambridge University Press,p.72-87, ISBN 978-0-521-84170-2
- Dugatkin, Lee, "Game Theory and Animal Behavior" 1998, p. 2-20, Oxford University Press, ISBN 0-19-509692-4
- Samuelson, L. 2002 Evolution and game theory. JEP. 16(2): 46-66.
- Weibull, J. W. (1995) Evolutionary game theory, MIT Press
- Dawkins, Richard, "The Selfish Gene", Oxford University Press, 1976, p. 76-78, ISBN 0-19-929114-4
- Maynard Smith, "Evolution and the Theory of Games", 1982, Cambridge University Press, p.28,ISBN 0-521-24673-3
- Maynard Smith, "Evolution and the Theory of Games", 1982, Cambridge University Press, p.33,ISBN 0-521-24673-3
- Parker and Thompson,"Dung Fly Struggle: a test of the War of Attrition",1980, "Behaviour. Ecology, Sociobiology", V.7, p.37-44
- Harman, O., "The Price of Altruism", 2010, Bodley Head Books, Ch. 9, ISBN 9781847920621
- Dugatkin, Alan "Principles of Animal Behavior" 2004, WW Norton& Co., p. 255-260, ISBN 0-393-97659-9
- Sigmund, Karl, Institute of Mathematics University of Vienna, "William D. Hamilton’s Work in Evolutionary Game Theory", Interim Report IR-02-019
- Brembs,B., "Hamilton's Theory", 2001, Academic Press, dol:10.106/rwgn.2001.0581
- "Brembs,B., "Encyclopedia of Genetics", Academic Press, 2001, p. 906, ISBN 978-0-12-227080-2
- Axelrod, R. and W.D. Hamilton (1981) The evolution of cooperation, Science 211, 1390-6.
- Hughes, Oldroyd, Beekman, Ratnieks, "Ancestral Monogamy Shows Kin Selection Is Key to the Evolution of Eusociality", Science, 2008 Vol. 320 no. 5880 pp. 1213-1216 DOI: 10.1126/science.1156108
- Thorne, B., “Evolution of Eusociality in Termites”, Annual Review of Ecological Sysems, 1997, 28:27-54
- Nowak,Tarnita, Wilson, "The evolution of eusociality" Nature, 466: 1057-1062
- Bourke, Andrew, "The validity and value of inclusive fitness theory", Royal Society Proceedings, http://rspb.royalsocietypublishing.org/content/278/1723/3313.short
- Okasha, Samir, "Evolution and the Levels of Selection",2006, Oxford University Press, ISBN 978-0-19-926797-2
- Pacheco1,Santos, Souza, Skyrms, "Evolutionary dynamics of collective action in N-person stag hunt dilemmas", Proceeding of Royal Society, doi:10.1098/rspb.2008.1126
- Axelrod, R. (1984). The Evolution of Cooperation. ISBN 0-465-02121-2
- Nowak and Sigmund,"Evolution of indirect reciprocity", Nature, Vol 437/27 Oct 2005, p.1293-1295
- Axelrod, Robert, "The Evolution of Co-operation", Penguin Books, 1984, ch. 1-4, ISBN 0-14-012495
- Alexander R., "The Biology of Moral Systems (Foundations of Human Behaviour)", 1987,ISBN 978-0202011745
- Nowak, "Evolution of indirect reciprocity by image scoring", Nature 393, 1998, p573-575
- Nowak and Sigmund, "The Dynamics of Indirect Reciprocity", Journal of Theoretical Biology, (1998) 194, p.561-574, Article jt980775
- Osborn, Martin, Introduction to Game Theory, 2004, Oxford Press, p. 393-403 ISBN 0-19-512895-8
- Research S.Riechert, P. Hammerstein, "Putting Game Theory to the Test, SCIENCE, Vol. 267, 17 March 1995
- Chen Z, Tan JY, Wen Y, Niu S, Wong S-M (2012) A Game-Theoretic Model of Interactions between Hibiscus Latent Singapore Virus and Tobacco Mosaic Virus. PLoS ONE 7(5): e37007. doi:10.1371/journal.pone.0037007
- Alonzo, S.H. and Sinervo, B. 2000. Mate choice games, context-dependent good genes, and genetic cycles in the side-blotched lizard, Uta stansburiana. Behavioral Ecology Sociobiology
- Karl Sigmund, " Interim Report IR-05-076 John Maynard Smith and Evolutionary Game Theory”, International Institute for Applied Systems Analysis, Dec. 2005
- Allesina and Levine , "A competitive network theory of species diversity", Proceedings of the National Academy of Sciences (USA), March 17, 2011
- Palllen, Mark, “The Rough Guide to Evolution”, Penguin, 2009, p.74, ISBN 13:978-1-85828-946-5
- Zahavi, A. (1975) Mate selection - a selection for a handicap. Journal of Theoretical Biology. 53: 205-214.
- Grafin, Alan, "Biological Signals as Handicaps, Journal of Theoretical Biology, 1990, Issue 144 p.517-546
- Pallen, M., "Rough Guide to Evolution", Penguin Books, 2009, p.123, ISBN 13:978-185823-946-5
- Matja, Szolnoki, "Coevolutionary games – a mini review", Biosystems, 2009
- Cliff and Miller, "Tracking the red queen: Measurements of adaptive progress in co-evolutionary simulations", European Conference on Artificial Life, p. 200–218, 1995
- Sevan, Ficici and Pollack, "Pareto optimality in coevolutionary learning", European Conference on Artificial Life, pages 316–325, 2001
- Bergstrom, C., M. Lachmann. 2003. The red king effect: when the slowest runner wins the coevolutionary race. Nat. Acad. Sci. 100: 593-598.
- Nowak, Martin, "Evolutionary Dynamics", 2006, Harvard University Press, p.152-154, ISBN 978-0-674-02338-3
- Nowak & Sigmund, "Evolution of indirect reciprocity", Oct 2005, Nature, Vol. 437/27
- Kaznatcheev, A. (2010, March). Robustness of ethnocentrism to changes in inter-personal interactions. In Complex Adaptive Systems–AAAI Fall Symposium.
- Grafen, A. (1990). "Biological signals as handicaps". Journal of Theoretical Biology 144 (4): 517–546. doi:10.1016/S0022-5193(05)80088-8. PMID 2402153.
- Jee, J.; Sundstrom, A.; Massey, S.E.; Mishra, B. (2013). "What can information-asymmetric games tell us about the context of Crick's 'Frozen Accident'?". Journal of the Royal Society Interface. 10 (88): 20130614. doi:10.1098/rsif.2013.0614.
- Interview of Maynard Smith by Richard Dawkins http://www.webofstories.com/play/7302?o=MS
- Soros, G (1987) The Alchemy of Finance (Simon & Schuster, 1988) ISBN 0-671-66238-4 (paperback: Wiley, 2003; ISBN 0-471-44549-5)
- Maynard Smith, J. (1982) Evolution and the Theory of Games.
- P. Hammerstein and R. Selten, "Game theory and evolutionary biology", in Handbook of Game Theory with Economic Applications, R. J. Aumann and S. Hart, Eds. (Elsevier, Amsterdam, 1994), vol. 2, pp. 929–993
- Hofbauer, J. and Sigmund, K. (1998) Evolutionary games and population dynamics, Cambridge University Press
- Taylor, P. D. (1979). Evolutionarily Stable Strategies with Two Types of Players J. Appl. Prob. 16, 76-83.
- Taylor, P. D., and Jonker, L. B. (1978). Evolutionarily Stable Strategies and Game Dynamics Math. Biosci. 40, 145-156.
- Web of Stories, Interview - Life and Work - John Maynard Smith by Richard Dawkins, http://www.webofstories.com/play/7301?o=S&srId=240151
- For the Interested Layperson
- Sigmund, Karl; "Games of Life", Penguin Books, also Oxford University Press, 1993, ISBN 0198547838
- Dawkins, Richard; "The Selfish Gene", Oxford University Press, ISBN 0-19-92114-4
- Davis, Morton,; "Game Theory – A Nontechnical Introduction", Dover Books, ISBN 0-486-29672-5
- Axelrod, Robert; "The Evolution of Co-operation", Penguin Books, 1990, ISBN 0-14-012495
- Historical perspective
- Kohn, Marek; "A Reason for Everything", Faber and Faber, ISBN 0-571-22393-1
- Harman, Oren; "The Price of Altruism", Bodley Head Books, 2010, ISBN 978-1-84792-062-1
- Segerstrale, Ullica; "Nature's Oracle - The life and work of W.D. Hamilton", Oxford University Press,2013, ISBN 978-0-19-860727-4
For the Serious Reader with Appreciation of Mathematics
- Maynard Smith, John; "Evolution and the Theory of Games", Cambridge University Press, ISBN 0-521-28884-3
- Dugatkin and Reeve; "Game Theory and Animal Behavior", Oxford University Press, ISBN 0-19-513790-6
- For the mathematically adept
- Nowak, Martin; "Evolutionary Dynamics", Harvard University Press, ISBN 978-0-674-02338-3
- Vincent and Brown; "Evolutionary Game Theory, Natural Selection and Darwinian Dynamics", Cambridge University Press, ISBN 0-521-84170-4
- Hofbauer and Sigmund; "Evolutionary Games and Population Dynamics", Cambridge University Press, ISBN 0-521-62570-X
- EGT at the Stanford Encyclopedia of Philosophy
- Evolving Artificial Moral Ecologies at The Centre for Applied Ethics, University of British Columbia
- Evolutionary game theory at the Open Directory Project