Journal of Theoretical Biology 497 (2020) 110288

Contents lists available at ScienceDirect

Journal of Theoretical Biology

journal homepage: www.elsevier.com/locate/jtb

Periodic orbit can be evolutionarily stable: Case Study of discrete

replicator dynamics
Archan Mukhopadhyay∗, Sagar Chakraborty
Department of Physics, Indian Institute of Technology Kanpur, Uttar Pradesh 208016, India

a r t i c l e i n f o a b s t r a c t

Article history: In evolutionary game theory, it is customary to be partial to the dynamical models possessing fixed points
Received 7 August 2019 so that they may be understood as the attainment of evolutionary stability, and hence, Nash equilibrium.
Revised 8 February 2020
Any show of periodic or chaotic solution is many a time perceived as a shortcoming of the correspond-
Accepted 15 April 2020
ing game dynamic because (Nash) equilibrium play is supposed to be robust and persistent behaviour,
Available online 18 April 2020
and any other behaviour in nature is deemed transient. Consequently, there is a lack of attempt to con-
Keywords: nect the non-fixed point solutions with the game theoretic concepts. Here we provide a way to render
Game theory game theoretic meaning to periodic solutions. To this end, we consider a replicator map that models Dar-
Evolutionary dynamics winian selection mechanism in unstructured infinite-sized population whose individuals reproduce asex-
Replicator map ually forming non-overlapping generations. This is one of the simplest evolutionary game dynamic that
Periodic orbits exhibits periodic solutions giving way to chaotic solutions (as parameters related to reproductive fitness
Evolutionary stability
change) and also obeys the folk theorems connecting fixed point solutions with Nash equilibrium. Inter-
estingly, we find that a modified Darwinian fitness—termed heterogeneity payoff—in the corresponding
population game must be put forward as (conventional) fitness times the probability that two arbitrarily
chosen individuals of the population adopt two different strategies. The evolutionary dynamics proceeds
as if the individuals optimize the heterogeneity payoff to reach an evolutionarily stable orbit, should it
exist. We rigorously prove that a locally asymptotically stable period orbit must be heterogeneity stable
orbit—a generalization of evolutionarily stable state.
© 2020 Elsevier Ltd. All rights reserved.

1. Introduction nomics so that the agents are not seen as merely uncompromis-
ing maximisers of profit but as driven by some sort of selection
The influence of Malthus and contemporary economists on process (Alchian, 1950; Friedman, 1953; Nelson and Winter, 2002;
the development of the theory of natural selection is well- Samuelson, 2002).
documented (Darwin, 1887); and so is very well known that it In a monomorphic population invaded by a tiny fraction of mu-
was Darwin (Darwin, 1871) who first gave a scientific argument tants, ESS either renders a higher expected payoff through its per-
for why the sex ratio in most sexually reproducing species is ap- formance against itself or, in case the strategy ties with that of
proximately 1:1 between males and females, or in modern game the mutant’s, it fetches more expected payoff against the mutant
theoretic parlance, why the sex ratio 1:1 is an evolutionarily sta- compared to what the mutant would. It is remarkable (Hofbauer
ble strategy (ESS) (Maynard Smith, 1972; Maynard Smith and and Sigmund, 2003) that a strict Nash equilibrium (NE) is ESS and
Price, 1973). The symbiotic relationship between economists and ESS must be NE. It just so happens that the condition for NE—
biologists (also, sociologists, ethologists, and related researchers) needed for existence of ESS—is enough to explain quite a few bio-
through the evolutionary game theory, thus, started with Darwin’s logical conflict scenarios. Hence, it not surprising that the concept
work and is still going very strong: While Dawkins (1989) remarks of the evolutionary stability has publicised the idea of NE for non-
that the evolutionary game theoretical concept of evolutionary sta- economists.
bility is one of the most important advances in evolutionary the- It may not be very wrong to remark that many refinements of
ory, the economists do advocate for evolutionary theorizing in eco- NE, that solely bank on the ideas related to rationality, are not
very satisfactory; at least, the rationality-based mechanism lead-
ing to them in a game definitely is not so. As a result, over the
∗
Corresponding author. Tel.: +91 8953433862. last thirty years or so, evolutionary models are slowly but surely
E-mail addresses: [email protected] (A. Mukhopadhyay), [email protected] (S. being preferred to rationality-based models. This is more so be-
Chakraborty).

https://doi.org/10.1016/j.jtbi.2020.110288
0022-5193/© 2020 Elsevier Ltd. All rights reserved.
2 A. Mukhopadhyay and S. Chakraborty / Journal of Theoretical Biology 497 (2020) 110288

cause now-a-days, rather than interpreting a game as an idealized fixed point (Börgers and Sarin, 1997; Hofbauer and Schlag, 20 0 0;
rational interaction, it is more sensible to interpret it as a model Vilone et al., 2011; Pandit et al., 2018). While one could say that
of an actual interaction wherein an equilibrium is seen as the re- the appearance of non-fixed point solutions is shortcoming of the
sult of a dynamic adjustment process. In this context, it is worth corresponding model and there should exist an evolutionary model
noting that the evolutionary game theory manifests itself through that unfailingly ensures convergence to NE, there is no strong logic
mathematical models that describe adaptions of the players’ be- to presume that actual behaviour would be in line with such a
haviours over the course of repeated plays of a game as a dynamic model.
process. Evolutionary stability concept is not only able to rule out In view of the above, in this paper, we take first step of propos-
some of the NEs in case there are more than one of them, it also ing the hitherto ill-understood question: Is there any game the-
unravels the non-rationality-based mechanism for the realizable oretic argument possible that we can connect with the periodic
NEs. For example, ESS is known (van Damme, 1991) to be both outcomes of the dynamical models in population games and what
proper (Nash) equilibrium (Myerson, 1978) and trembling hand could be the physical implication of the periodic outcome? Given
perfect (Nash) equilibrium (Selten, 1975). Thus, if one sees the at- the plethora of models of evolutionary dynamics, we decide to
tainment of the equilibrium as a consequence of an evolutionary- work with the replicator map adapted for the two-player-two-
based mechanism, then one is no longer faced with the difficultly strategy games (Pandit et al., 2018) because of the following con-
of explaining why rational players should tremble in a rationality- crete reasons: (i) this evolutionary game dynamic models Dar-
based model. winian selection, (ii) relationship of its fixed points with NE and
The most used mathematical model in the evolutionary game ESS is well established through standard folk theorems and related
theory is without doubt the replicator equation (Taylor and Jonker, theorems, and (iii) it is able to show periodic solutions (that bifur-
1978)—a highly simplified model of selection and replication. There cate into chaotic solution). Since our main aim is to find game the-
are folk theorems connecting stable fixed points of the replica- oretic connection for periodic orbits, we restrict our analyses only
tor dynamics with static solution concepts of noncooperative game to the mixed strategy domain. This is so because any pure state is
theory played by rational players (Cressman and Tao, 2014). It en- a fixed point of the replicator map.
ables a biologist to predict the dynamical outcome by finding the We end up convincingly showing that it is very much pos-
Nash equilibrium (NE) of the corresponding one shot game. This is sible to interpret locally asymptotically stable periodic orbits as
the case for other monotone dynamics in evolutionary game the- representing evolutionarily stable scenario in the setting of a re-
ory as well. In literature there are many different evolutionary dy- peated game. All one needs to do is to appropriately generalize
namical models: some of them can be obtained by varying revision the concepts of ESS and NE such that the individuals in the pop-
protocol (Lahkar and Sandholm, 2008; Hofbauer and Sandholm, ulation game appear to be optimizing their way to ‘survival of the
2011), e.g., replicator dynamics (Taylor and Jonker, 1978), best re- fittest’. We find that the effective ‘fitness’ in the game must be (re-
sponse dynamics (Gilboa and Matsui, 1991), Brown–von Neumann– )interpreted as fitness multiplied by the probability that two arbi-
Nash dynamics (Brown and von Neumann, 1950), Smith dynam- trarily chosen members of the population belong to two different
ics (Smith, 1984) and logit dynamics (Blume, 1993); whereas some phenotypes.
can be seen as variants of incentive dynamics (Harper and Fryer, However before embarking on the technical discussion of the
2015), e.g., replicator dynamics (Taylor and Jonker, 1978), best re- repeated games and the corresponding game theoretical concepts
sponse dynamics (Gilboa and Matsui, 1991), logit dynamics (Blume, of equilibrium, without further ado we revisit the dynamics of the
1993) and projection dynamics (Nagurney and Zhang, 1997). All replicator map that we have used as the paradigmatic model in
the aforementioned dynamics have the common property of con- this paper.
verging towards NE (Sandholm et al., 2008; Hofbauer et al., 2009;
Feldman et al., 2017; Pandit et al., 2018). It has also been shown
in the literature that evolutionarily stable state can be related 2. Periodic orbits in replicator map
to locally asymptotically stable fixed point for both replicator
map (Pandit et al., 2018) and replicator flow (Cressman and Tao, Consider the population game where there is an underlying
2014). normal form game with, say, N pure strategies and an N × N payoff
Although it is not justified to argue that equilibrium play should matrix U. Any mixed strategy is an element of the corresponding
explain the outcomes of every games in every possible scenario, simplex  N . Subsequently, one could define a population game on
this idea of equilibrium being connected with convergence to the  n between n (pheno-)types with fractions x1 , x2 , , xn such that
fixed point is so deep-rooted that it is commonly held that any every type can be mapped to a particular strategy in  N ; say, ith
non-equilibrium behaviour is necessarily transient, and only equi- type in the population game is realized as strategy pi ∈  N . An el-
ement, π ij , of n × n payoff matrix Π (say) of the population game
librium behaviour is persistent and robust to be ultimately real- 
ized. As a consequence, it appears very intriguing when the pos- is pi · Up j and the fitness of ith type is given by (Πx )i = πi j x j .
sibilities of non-convergence to NE solutions show up as robust Connection between the two aforementioned simplices is that the
dynamics of state x ∈  n induces a dynamics for average popula-
asymptotic solutions—and not as transients—in evolutionary mod- 
els. There is no known way (such as folk theorems) to predict such tion strategy p̄ = ni=1 pi xi on  N .
non-NE outcomes of dynamics from the knowledge of one-shot In classical game theory, rational players optimize their payoff
noncooperative game theory. following the concept of NE strategy profile wherein the strategies
Few examples of such non-fixed point robust asymptotic of the players are best responses to each other. Hence, the underly-
dynamical solutions in evolutionary game theory are as fol- ing game with payoff matrix U has a mixed NE (pˆ ) which is math-
lows: Replicator dynamics, Brown–von Neumann–Nash dynam- ematically defined as
ics, and Smith dynamics can show limit cycle as possible out- pˆ T Upˆ = pT Upˆ , ∀ p ∈ N . (1)
come (Hofbauer and Sandholm, 2011). Chaotic behaviour has been
found in replicator dynamics (Skyrms, 1992; Sato et al., 2002) and In evolutionary game theory the concept of ESS plays the cen-
Brown–von Neumann–Nash dynamics (Waters, 2009b). Logit dy- tral role as it ensures that a population adopting this strategy can’t
namics, the noisy version of best response dynamics (Ferraioli, be invaded by any infinitesimal fraction of mutants adopting an al-
2013), can also lead to periodic solutions. Replicator map can show ternative strategy. Mathematically, the strategy pˆ is ESS of the un-
both chaotic and periodic outcomes along with convergence to derlying game if there exists a neighbourhood Bpˆ of pˆ such that
 A. Mukhopadhyay and S. Chakraborty / Journal of Theoretical Biology 497 (2020) 110288 3

∀p ∈ Bpˆ \pˆ the following inequality holds: periodic orbits, and chaotic trajectories (Pandit et al., 2018): The
map, in general, has two boundary fixed points, x = 0 and 1, and
pˆ T Up > pT Up. (2) one interior fixed point, x = S/(S + T − 1). The interior fixed point
It is straightforward to show that ESS implies NE. of this map is stable when S(T − 1 )/(S + T − 1) < 2 and undergoes
Along the line of the above discussion the idea of NE and ESS flip bifurcation at S(T − 1 )/(S + T − 1) = 2, giving rise to a two-
can be extended to the population game with payoff matrix Π. period orbit. Subsequently, as one drives S(T − 1 )/(S + T − 1) fur-
However now the (pheno-)types are the possible strategies. Hence ther away from 2, a period doubling cascade—giving rise to higher
it is defined in terms of state of the population consisting those period orbits and ultimately chaos—is observed.
phenotypes. Specifically, for symmetric population game with pay- Now, let’s assume that a sequence of states, {xˆ (k ) : xˆ(k ) ∈
off matrix Π, if xˆ is the mixed NE (state) then (0, 1 ), k = 1, 2, · · · , m}, with xˆ (i) = xˆ ( j ) ∀i = j, represents a periodic
orbit with prime-period m. Then for any k ∈ {1, 2, , m}, by con-
xˆ T Πxˆ = xT Πxˆ , ∀ x ∈ n . (3)
struction, we have
The state xˆ is ESS (evolutionarily stable state) of the population  
xˆ(k+1) = xˆ(k ) + xˆ(k ) (1 − xˆ(k ) ) (Πxˆ (k ) )1 − (Πxˆ (k ) )2 , (8)
game if there exists a neighbourhood Bxˆ of xˆ such that ∀x ∈ Bxˆ \xˆ
the following inequality holds, where naturally, xˆ (m+1 ) = xˆ (1 ) . Summing all the m expressions im-
T T
xˆ Πx > x Πx. (4) plied by Eq. (8), we get

Again, ESS implies mixed NE.

m  
xˆ(k ) (1 − xˆ(k ) ) (Πxˆ (k ) )1 − (Πxˆ (k ) )2 = 0. (9)
The replicator map maps frequency of a phenotype in kth gen-
k=1
eration to its frequency in (k + 1 )th non-overlapping generation.
Specifically, the map is expressed in the following form (Börgers It is interesting to note that 2xˆ(k ) (1 − xˆ(k ) ) is the probability that
and Sarin, 1997; Hofbauer and Schlag, 20 0 0; Vilone et al., 2011; two arbitrarily chosen members of the population belong to two
Pandit et al., 2018): different phenotypes. In population genetics of the simple case
  of one-locus-two-allele, under Hardy–Weinberg assumptions, the
(k )
xi(k+1) = fi (x ) = xi(k ) + xi(k ) (Πxˆ )i − xˆ (k)T Πxˆ (k) . (5) analogous expression is called heterozygosity that measures the
proportion of heterozygous individuals in the population (Rice,
Here xi(k ) is the frequency of ith phenotype in the population of kth 1961). For future convenience, we denote 2xˆ(k ) (1 − xˆ(k ) ) by Hxˆ (k)
generation and x(k ) = (x1(k ) , x2(k ) , · · · , xn(k ) ). Π is the payoff matrix of and call it heterogeneity as it is a measure of how heterogeneous-
the corresponding one shot game associated with the dynamics. In strategied the population is.
general, the map can give rise to unphysical solutions, i.e., xi ∈ [0,
1] for some i; therefore, not all n × n payoff matrices are physically 3. Extension of NE and ESS
allowed.
For two-player-two-strategy game i ∈ {1, 2} and Π is a 2 × 2 ma- Our ultimate aim concerns with relating the dynamical out-
trix. On rewriting x1(k ) = x(k ) and, hence, x2(k ) = 1 − x(k ) , Eq. (5) be- comes with the corresponding population game theoretic out-
comes, comes, we must first discuss how we can extend the existing
    framework of equilibrium states for a set of states (containing m
x(k+1) = f x(k ) = x(k ) + x(k ) (1 − x(k ) ) (Πxˆ (k ) )1 − (Πxˆ (k ) )2 . (6)
elements) that may be periodic orbit. Analogously, we want to
The fixed point of this replicator map given by Eq. (6) is connected study the scenario of m-period games (with payoff matrix U) that
with game theoretic outcomes through folk theorems. The NE state is an extensive form of game where a base game is played m times.
are the fixed point of this map whereas any locally asymptotically Unless otherwise specified, for the sake of simplicity, we discuss
stable interior fixed point is known to be ESS (Pandit et al., 2018). the case of two-player-two-strategy game (i.e., N = 2) throughout
By the concept of strong stability (Hofbauer and Sigmund, 2003) it the paper (see, however, Appendix A). The caveat we must keep in
can also be shown that if the average strategy of the population mind is that while a standard m-period game is usually all about
in the underlying game asymptotically converges to the strategy maximising (given the belief about the opponent) the total payoff
adopted by a phenotype in the undelying game then that strategy accumulated over all the stages, the m-period games we are inter-
must be ESS of the underlying game. Hence, in a way one can con- ested are in the context of the replicator map where the dynamics
nect the dynamics of this map with the game theoretic outcome at each stage (generation) is driven by the payoffs of immediately
of the underlying game. preceding stage. It already gives us hint that players playing m-
For game theoretic studies, it suffices to work with the follow- period game in line with the replicator map need not be optimiz-
ing form of 2 × 2 payoff matrix: ing the accumulated payoff. So what do they optimize?
1 S
= ; S, T ∈ R, (7) 3.1. Heterogeneity equilibrium
T 0
as it is capable of representing all the twelve ordinal classes of In classical game theory, rational players optimize their payoff
symmetric games found in the standard literature. The dynamics following the concept of NE strategy profile wherein the strategies
of replicator map for this form of payoff matrix has been studied of the players are best responses to each other. However, rational-
in literature and specific conditions for which this map gives strict ity is a redundant concept in evolutionary dynamics that is gov-
physical solutions are known (Pandit et al., 2018). The reason be- erned by, say, Darwinian selection. While a posteriori justification
hind choosing this game to be symmetric is that we assume, as is is furnished later in the form of a successful self-consistency, we
norm in the evolutionary game theory, that (i) the players’ strat- propose to study a scenario where rather than the expected payoff,
egy sets are identical, (ii) the payoff received by a player playing heterogeneity weighted expected payoff (or simply heterogeneity
against an opponent doesn’t change with the identities of the play- payoff, for the sake of brevity) is being optimized. By heterogene-
ers, and (iii) players don’t make their choices of strategy based on ity payoff we merely mean that payoff is weighted (multiplied)
features of the opponent. by the heterogeneity defined using opponent’s mixed strategy, p(k)
The simple map given, by Eq. (6), has rich dynamical properties (say), i.e., Hp(k) ≡ 2 p(k ) (1 − p(k ) ). Here p(k ) = ( p(k ) , 1 − p(k ) ) is the
showing wide range of asymptotic behaviours, e.g., fixed points, completely mixed strategy of the opponent. Note that now we are
4 A. Mukhopadhyay and S. Chakraborty / Journal of Theoretical Biology 497 (2020) 110288

using strategy, and not state, to define heterogeneity. We reiterate

that we work only in the completely mixed strategy domain as our
m  
Hxˆ (k) (Πxˆ (k ) )1 − (Πxˆ (k ) )2 = 0. (14)
main intention is to justify the emergence of the periodic orbits (of
k=1
prime period more than one) which must be totally mixed states
as all the pure states are fixed points of the replicator map. We note that a set of states formed by a single mixed NE repeated
The condition of mixed NE as expressed by Eq. (1) can be triv- m times becomes a trivial HO(m) if we remove the restriction im-
ially rewritten as, posed by xˆ (i ) = xˆ ( j ) for i = j. However, whether non-trivial solutions
    to Eq. (14) exist depends on the exact structure of the payoff ma-
Hpˆ pˆ T Upˆ = Hpˆ pT Upˆ ; 0 < Hpˆ ≤ 0.5. (10) trix Π. Furthermore, on comparing Eq. (14) with Eq. (9), one notes
that m-period orbit of replicator map must be HO(m) and vice versa.
It is clear that Eq. (1) considers expected payoff as incentive while
Eq. (10) considers heterogeneity payoff as incentive. Obviously, 3.3. Heterogeneity stable orbit
they both have mixed NE as a unique solution should it exist, im-
plying that a mixed NE provides indifference in heterogeneity pay- Having generalized mixed NE to HO, in this subsection we ask
off for unilateral deviation in case of 1-period game. what the generalization of evolutionary stability is and how that
We contextually propose that there exists an equilibrium generalization of ESS will relate to HO. In evolutionary game the-
among the mixed strategies for m-period games, that we name ory the concept of ESS plays the central role as it ensures that a
as heterogeneity weighted Nash equilibrium or heterogeneity equi- population in this state can’t be invaded by an infinitesimal frac-
librium (HE) for brevity. The HE(m) strategy profile (where m de- tion of mutant having some alternative state. As the stable fixed
notes that m-period game is under consideration) consists of pairs points of the replicator map are ESS, so it can be claimed that nat-
of strategies that are best responses to each other in the follow- ural selection alone is sufficient to stop invasion by any alternative
ing sense: Assuming that a player plays the strategy of any stage state once the population is fixed at ESS. Now, given the idea of
of the m-period in all the m stages of play with its opponent, the heterogeneity payoff, is there any state profile for m-period orbit,
player cannot get more accumulated heterogeneity payoff by devi- that is resilient against an infinitesimal mutant fraction? ESS xˆ is
ating unilaterally. Mathematically, we have the following: the state of the population that is resilient against an infinitesi-
Definition: The sequence of strategies {pˆ (k ) : pˆ (k ) ∈ (0, 1 ), k = mal mutant fraction— fraction with any state x(m ) = xˆ ; in formal
1, 2, · · · , m} over m-period game is an HE(m) if ∀j ∈ {1, 2, , m}, mathematical notations,
m   m   xˆ T Π[(1 −  )xˆ +  x(m ) ] > xT(m ) Π[(1 −  )xˆ +  x(m ) ] (15)
Hpˆ (k) pˆ ( j )T Upˆ (k ) = Hpˆ (k) pT Upˆ (k ) ; ∀ p ∈ ( 0, 1 ). (11)
for   1. Now, let’s define x ≡ (1 −  )xˆ +  x(m ) . One can construct
k=1 k=1
neighbourhood Bxˆ of xˆ such that x ∈ Bxˆ \{
xˆ }. Multiplying
 both sides
We have already observed that mixed NE profile is the unique of Inequality (15) by  and adding 1 −  xˆ T Πx to both the sides,
HE profile for 1-period game. A closer look reveals that any we arrive the condition of ESS given by Inequality (4).
single mixed NE profile played over all the stages of the m- We can rewrite Inequality (4) as,
   
period game induces an HE(m): of course, if Eq. (11) holds then Hx xˆ T Πx > Hx xT Πx , (16)
d m T ˆ (k ) ] = 0, implying
k=1 Hpˆ (k ) [p Up
dp whenever x in the neighbourhood Bxˆ , i.e., x ∈ Bxˆ \{xˆ }. Inequal-
m   ity (16) is another equivalent definition of ESS. In line with the
Hpˆ (k) (Upˆ (k ) )1 − (Upˆ (k ) )2 = 0. (12) concept of HO, we propose heterogeneity weighted evolutionarily
k=1 stable orbit—for brevity, heterogeneity stable orbit, HSO(m)—as an
It is now easily comprehensible that a set of strategies that is es- extension of ESS.
sentially a single mixed NE repeated m times is an HE(m). To see Definition: HSO(m) of a map—xi(k+1 ) = g(xi(k ) )—is a sequence
it more transparently, we recall that the term in the third bracket of states, {xˆ (k ) : xˆ(k ) ∈ (0, 1 ); k = 1, 2, · · · , m; xˆ (i ) = xˆ ( j ) ∀i = j} such
in Eq. (12) vanishes individually if pˆ (k ) is mixed NE strategy. More that
m m
interesting, however, is the non-trivial scenario when Eq. (12) is
fulfilled by a set of strategies that is not a single mixed NE re- Hx(k) xˆ (1)T Πx(k ) > Hx(k) x(1)T Πx(k ) , (17)
k=1 k=1
peated m times. However, whether such solutions exist depends
on the exact structure of the payoff matrix. We note that, if n is a for any orbit {x(k ) : x(k ) ∈ (0, 1 ); k = 1, 2, · · · , m} of the map starting
(1 )
multiplicative factor of m, then the set of strategies forming HE(n) in some infinitesimal neighbourhood B (1) \{xˆ } of xˆ (1 ) .
xˆ
repeated m/n times form an HE(m). It is easy to observe that mixed ESS is the unique HSO(1). In
passing, we remark that the concept of incentive stable state equi-
3.2. Heterogeneity orbit librium (Harper and Fryer, 2015) to describe incentive dynamics is
simply HSO(1). We also observe that one could in principle replace
The concept of HE(m) in repeated games (with payoff matrix states by strategies and use appropriate payoff matrix (U, say) in
U) can be adapted to define an equilibrium using a set of states Inequality (17) to analogously define heterogeneity stable strategy
for a population consisting of two types (with payoff matrix Π). (HSS).
Hence, in what follows, we propose heterogeneity weighted Nash Since we know that ESS serves as a refinement of NE, it would
equilibrium orbit or heterogeneity orbit (HO) for brevity. be rather satisfying if HSO(m) serves as a refinement of HO(m).
Definition: The sequence of states {xˆ (k ) : xˆ(k ) ∈ (0, 1 ), k = For m = 1, the sought refinement is mere tautology because the
1, 2, · · · , m} where xˆ (i ) = xˆ ( j ) for i = j, is an HO(m) if ∀j ∈ {1, 2, , concepts of HSO and HO boil down to the concepts of ESS and
m}, NE respectively. For the case of any general m, owing to Inequal-
m m
ity (17) there exists a neighbourhood Nxˆ(1) of xˆ(1 ) in (0,1) such
Hxˆ (k) [xˆ ( j )T Πxˆ (k ) ] = Hxˆ (k) [xT Πxˆ (k ) ]; ∀x ∈ ( 0, 1 ). (13) that ∀ x(1 ) ∈ Nxˆ(1) \{xˆ(1 ) } (where B (1) = Nxˆ(1) × (0, 1 )), the follow-
xˆ
k=1 k=1
ing holds:

We have already observed that mixed NE profile is the unique

  m    
d m T x (k ) ] = 0, implying
x(1) − xˆ(1) Hx(k) Πx(k ) − Πx(k ) < 0, (18)
HO(1). If Eq. (13) holds then dx 1 2
k=1 Hxˆ (k ) [x Πˆ k=1
 A. Mukhopadhyay and S. Chakraborty / Journal of Theoretical Biology 497 (2020) 110288 5

The importance of this theorem is akin to that of the folk theo-

m     rems: One can deduce on the asymptotic periodic outcome of the
⇒ lim Hx(k) Πx(k ) − Πx(k ) = 0, (19) replication-selection dynamics by studying the payoff matrix of the
x(1) −xˆ(1) →0 1 2
k=1 game keeping in mind the concept of HSO. Thus, we believe that
this representative theorem has far reaching implications on the
m   study of evolutionary dynamics. This theorem enables one to un-
⇒ Hxˆ (k) (Πxˆ (k ) )1 − (Πxˆ (k ) )2 = 0. (20) derstand what the game-theoretic interpretation of a robust stable
k=1 periodic orbit is. Recall that periodic orbits are a common occur-
Comparing Eq (20) with Eq. (14) we conclude that HSO(m) implies rence in many dynamical systems of evolutionary game theory.
HO(m).
Henceforth, unless otherwise specified, all further discussions
involve only HSO(m) of replicator map (cf. Appendix B). 5. Strongly stable strategy set

4. HSO and dynamical stability Though our study has associated periodic orbit with evolution-
arily stability, we lack the corresponding insight in the underlying
It is clear that HSO(1) is nothing but evolutionarily stable state normal form game where a particular strategy corresponds to a
and it has been shown in literature that locally asymptotically sta- particular (pheno-)type in the population game. We already know
ble fixed point of the replicator map is HSO(1) (Pandit et al., 2018). that unlike the continuous time dynamics, ESS or HSO(1) need not
We emphasize that HSO(1) is locally asymptotically stable fixed be the stable fixed point of replicator map. As an example, Leader
point even for the replicator equation (Cressman and Tao, 2014). game can lead to periodic or chaotic outcome even though it pos-
It, thus, is very natural to suspect that there must be a connection sesses ESS (Pandit et al., 2018). It hints that the average population
between stable periodic orbit of period m and HSO(m). In fact, the strategy in normal form game also don’t converge to any particular
following theorem tells us that so is the case: strategy that happen to be evolutionary stable strategy of the cor-
Theorem: If the sequence of states {xˆ (k ) : xˆ (k ) ∈ int2 ; k = responding normal form game. It, thus, is important to understand
1, 2, · · · , m}, where xˆ (i ) = xˆ ( j ) for i = j, is a locally asymptotically sta- the dynamics from the point of view of underlying normal form
ble m-period orbit of the replicator map for two-player-two-strategy game.
game, then it must be HSO(m). Our studied population game corresponds to two types, i.e.,
Proof: Let sequence of states {xˆ (1 ) , xˆ (2 ) , · · · , xˆ (m ) } be a locally n = 2. Let’s consider that the type with frequency x is using strat-
asymptotically stable periodic orbit of the replicator map given in egy p1 and the other type with frequency 1 − x is using strat-
Eq. (6). Then, by the definition of period orbit, each of the state egy p2 where both p1 , p2 ∈  N . The average population strategy at
from the set must be a fixed point of the map fm (x). We assume kth generation is given as, p̄(k ) = x(k ) p1 + (1 − x(k ) )p2 ∈ N . Hence
that the states are arranged in temporal order. Since we have as- we can rewrite the condition such that the sequence of states
sumed local asymptotic stability, by construction, ∃ a neighbour- {xˆ (k) : xˆ(k) ∈ (0, 1 ); k = 1, 2, · · · , m} is a m-periodic orbit of repli-
hood Nxˆ(1) of xˆ(1 ) in (0,1) such that ∀ x(1 ) ∈ Nxˆ(1) \{xˆ(1 ) } we have, cator map (refer Eq. (9)) in terms of undelying normal form game
in the following form,
|| f m (x(1) ) − xˆ(1) || m  
< 1. (21) (k ) (k )
||x(1) − xˆ(1) || Hxˆ (k) p1 .Up̄ − p2 .Up̄ = 0, (25)
k=1
Recalling f (x( j ) ) = x( j+1 ) and using the explicit form of the repli-
cator map, the above inequality can be rewritten as, (k )
where p̄ = xˆ(k ) p1 + (1 − xˆ(k ) )p2 . Hence, the average population
m   (k ) (k )
||x(1) − xˆ(1) + 1
2 k=1 Hx(k ) (Πx
(k ) ) − (Πx(k ) ) ||
1 2 strategy traverses through the sequence of strategies {p̄ : p̄ ∈
< 1. (22)
||x(1) − xˆ(1) || (0, 1 ); k = 1, 2, · · · , m} periodically.
(k ) (k ) 
Here, |||| stands for an appropriate norm which we can Definition: A sequence of strategies {p̄ : p̄ = 2i=1 xˆi(k ) pi ;
(i ) ( j)
conveniently take as the Euclidean norm. Inequality (22) im- xˆ(k ) ∈ (0, 1 ) ∀k = 1, 2, · · · , m; pi ∈ N } where p̄ = p̄ ∀i = j is
plies that x(1 ) − xˆ(1 ) must have a sign that is opposite to that strongly stable strategy set (SSSS(m)) if any initial average popula-
m  (k ) 
of k=1 Hx(k ) (Πx )1 − (Πx(k) )2 . Therefore, ∀x(1) ∈ Bxˆ (1) \{xˆ (1) } tion strategy p̄(k ) , that is sufficiently close to SSSS(m), converges to
where B (1) = Nxˆ(1) × (0, 1 ), SSSS(m).
xˆ (k )
  m   Theorem: If {p̄ : k = 1, 2, · · · , m} is SSSS(m) then {xˆ (k ) : k =
x (1 ) − xˆ(1)
k=1 Hx(k ) (Πx
(k ) ) − (Πx(k ) )
2 < 0
1 1, 2, · · · , m} is HSO(m).
m  ( 1 ) T ( k )  m  ( 1 )T ( k )  (k )  
⇒ k=1 Hx(k) xˆ Πx > k=1 Hx(k) x Πx . (23) Proof: By definition, p̄ = xˆ(k ) p1 + 1 − xˆ(k ) p2 . Any infinitesi-
mal perturbation around an element of SSSS(m) can be represented
Comparing this expression with Eq. (17) it is clear that it is noth- (k )    
ing but the condition for HSO(m). Hence, locally asymptotically sta- as p̄ +  (p1 − p2 ) = xˆ(k ) +  p1 + 1 − xˆ(k ) −  p2 where  → 0.
ble periodic orbits of period m are HSO(m). Q.E.D. Thus, we note that if any initial average population strategy is suf-
The converse of this theorem is not true, i.e., an HSO(m) ficiently close to an element of SSSS(m), then in the population
need not always be a locally asymptotically stable periodic or- dynamics the initial state is sufficiently close to the correspond-
bit of period m. Inequality (23) is only a necessary condition ing element of the sequence of states {xˆ (k ) : k = 1, 2, · · · , m}. Since
for the fulfilment by the definition the initial average population strategy converges
  of(k)Inequality(k) (22)
 ; one additionally requires
( 1/2 ) mk=1 Hx(k ) (Πx )1 − (Πx )2 < 2|x(1) − xˆ(1) | for the con- to SSSS(m), if we start sufficiently close to any state of the se-
verse to hold true. Therefore, a HSO(m), that also happens to be quence {xˆ (k ) : k = 1, 2, · · · , m}, the population state must converge
an orbit, is a locally asymptotically stable m-periodic orbit of the to this set. Hence, the set of states is locally asymptotically stable
replicator map if and only if m-periodic orbit that must be HSO(m) in line with the theorem
proved in Section 4. The converse of the theorem does not always
1
m   2 hold good as an HSO(m) need not be locally asymptotically stable
0< Hx(k) xˆ (1)T Πx(k ) − x(1)T Πx(k ) < 2|x(1) − xˆ(1) | . (24)
2 m-periodic orbit.
k=1
6 A. Mukhopadhyay and S. Chakraborty / Journal of Theoretical Biology 497 (2020) 110288

6. Illustrative examples

In order to make the concepts introduced in this paper more

accessible, we now take the examples of three games, viz., Pris-
oner’s Dilemma, Battle of Sex, and Leader game, where we con-
fine ourselves to the case of m = 2. Out of the possible physi-
cal solutions—fixed points and prime 2-periodic orbits—of equation
f 2 (x ) = x, we consider only the prime 2-period solutions as they
may be connected to HO(2) and HSO(2).
In the subsections to follow, we are specifically going to elab-
orate the following points in the context of the aforementioned
games:
• In case the dynamical outcome of a game is a 2-period orbit, Fig. 1. Locally asymptotically stable 2-period orbit, (xˆ(1) , xˆ(2) ) ≈ (0.52, 0.72 ), of Bat-
(xˆ(1) , xˆ(2) ), then that periodic orbit must be HO(2) defined by tle of Sex game is HSO(2). In subplot (a) blue solid curve and black dashed curve
respectively represent F1 and F2 for (0.52,0.72) plotted against  . Subplot (b) depicts
Eq. (14). By definition of HO(2) given by Eq. (13) any unilateral F1 vs.  for (xˆ(1) , xˆ(2) ) = (xˆ, xˆ) where xˆ is mixed NE (xˆ ≈ 0.61).
deviation by a player does not fetch more accumulated hetero-
geneity payoff when played against the periodic orbit.
• We know that a locally asymptotically stable 2-period orbit
must be HSO(2) as defined by Eq. (17). In order to check this,
it is convenient to define
1
2  
Fj≡ Hx(k) xˆ ( j )T Πx(k ) − x( j )T Πx(k ) , j ∈ {1, 2}; (26)
2
k=1

and observe that {xˆ (k ) : k = 1, 2} is HSO(m) if Fj > 0 for any

x( j ) = xˆ( j ) +  where || < ¯ for some positive ¯ ≤ 1.
• Furthermore, if an unstable 2-periodic orbits is HSO(2), it must
violate Inequality (24), i.e., Fj < 2| |2 does not hold true.
Without any loss of generality, the above points can easily be
adapted to find HE and HSS of any two-player-two-strategy game Fig. 2. Locally asymptotically stable 2-period orbit, (xˆ(1) , xˆ(2) ) ≈ (0.22, 0.68 ), of
Leader game (S = 5.0, T = 6.5) is HSO(2). In subplot (a) blue solid curve and black
(with payoff matrix U).
dashed curve respectively represent F1 and F2 for (0.52,0.72) plotted against  . Sub-
plot (b) depicts F1 vs.  for (xˆ(1) , xˆ(2) ) = (xˆ, xˆ) where xˆ is mixed NE (xˆ ≈ 0.48).
6.1. Prisoner’s dilemma

We consider the form of payoff matrix given by Eq. (7) where (xˆ(1) , xˆ(2) ) ≈ (0.22, 0.68 ) which is locally asymptotically stable.
S = −0.5 and T = 2.0 stands for Prisoner’s Dilemma game. The dis- Hence, (0.22,0.68) is an HO(2). As before, the accumulated hetero-
crete replicator dynamics of this game doesn’t have any physical 2- geneity payoff of each of the three states xˆ (xˆ ≈ 0.48 ), xˆ (1 ) , and
periodic orbit (Pandit et al., 2018). As periodic orbit must be HO(2), xˆ (2 ) when played against the periodic orbit is same ( ≈ 1.20).
we remark that this game doesn’t have any HO(2). By definition, Being locally asymptotically stable, the 2-period orbit must be
HSO(2) must be HO(2). Hence, this game doesn’t have any HSO(2) HSO(2). We confirm this through Fig. 2a where we observe that
either. F1, F2 > 0. Additionally, Fig. 2b stresses the fact that the mixed
NE repeated over two generations doesn’t satisfy the condition of
6.2. Battle of sex
HSO(2).
S = 5.5 and T = 4.5 (refer Eq. (7)) makes for the payoff ma-
trix of Battle of Sex game. Dynamics of this game has only one 6.3.2. Case II
physical 2-period orbit given by (xˆ(1 ) , xˆ(2 ) ) ≈ (0.52, 0.72 ). Hence, Another payoff matrix of Leader game may be realized
(0.52,0.72) is HO(2). by setting S = 7.5 and T = 8.0. This game has interest-
As implied by the definition of HO(2), even mixed NE xˆ (xˆ ≈ ingly three physical 2-period orbits, given by (xˆ(1 ) , xˆ(2 ) ) ≈
0.61) must fetch same accumulated heterogeneity payoff as any (0.12, 0.73 ); (0.36, 0.89 ); (0.14, 0.80 ). Hence, (0.12, 0.73); (0.36,
other arbitrary state, when played against the periodic orbit. It is 0.89); and (0.14,0.80) are HO(2). For these three HO(2), the ac-
indeed the case: We compare the accumulated heterogeneity pay- cumulated heterogeneity payoffs are approximately 1.25, 1.36,
offs for each of the three states xˆ , xˆ (1 ) , and xˆ (2 ) when played and 1.17 respectively, irrespective of what state plays against the
against the periodic orbit. They come out to be approximately 1.24. corresponding periodic orbits; as before we have checked this fact
This periodic orbit further happens to be locally asymptotically using the three states xˆ (xˆ1 ≈ 0.52; mixed NE), xˆ (1 ) , and xˆ (2 ) .
stable and hence it must be HSO(2). This is indeed the case as de- However, all of the three 2-period orbits are unstable. Some or
picted in Fig. 1a which shows that F1 > 0 and F2 > 0. This finding all of them may be HSO(2) that requires F1, F2 > 0. In Fig. 3a,b we
is non-trivial in the sense that even the mixed NE repeated twice note that (0.12,0.73) and (0.36,0.89) are HSO(2), and they also vi-
doesn’t satisfy the condition of HSO(2). This is showcased in Fig. 1b olate Inequality (24) as expected. The remaining periodic orbit is
where F1 < 0. This fact is in accordance with the fact that the NE not an HSO(2) as seen in Fig. 3c where F1, F2 < 0. Yet again, we
is an unstable fixed point. observe in Fig. 3d that the mixed NE repeated twice doesn’t sat-
isfy the condition of HSO(2).
6.3. Leader game
7. Discussion and conclusion
6.3.1. Case I
S = 5.0 and T = 6.5 gets us the payoff matrix of the Leader We remind ourselves that evolutionary game dynamics have
game. This game has only one physical 2-period orbit given by been successfully used to model real life problems in diverse fields,
 A. Mukhopadhyay and S. Chakraborty / Journal of Theoretical Biology 497 (2020) 110288 7

replicator dynamic—the replicator map for two-player-two-strategy

game—that is known to not only possess periodic and chaotic or-
bits, but also satisfy the folk theorems connecting fixed point so-
lutions to NE. We, then, introduce the concepts of new equilibria—
termed HE and HSS—in the context of m-period games, and de-
fine them in terms of the states of the population to introduce the
concepts of HO and HSO respectively. We can summarize the main
mathematical results in the following points: (i) HSO must be HO
(and similarly, HSS must be HE), (ii) a periodic orbit of replicator
map must be HO, and (iii) a locally asymptotically stable periodic
orbit is HSO. Thus, one is enabled to predict dynamical outcome
just by studying the payoff matrix of corresponding one shot game
even when the dynamic outcome is a period solution—this is a
clear development over the standard folk theorems for the repli-
cator dynamics.
What, however, is even more intriguing is that the replica-
tor dynamics, or in more fashionable terms, Darwinian selection
is such that it may not be the expected payoff or fitness that
Fig. 3. Unstable 2-period orbits of Leader game may or may not be HSO(2). Blue individuals optimize. Rather the fitness weighted by heterogeneity,
solid curve and black dashed curve respectively represent F1 and F2, and red solid termed heterogeneity payoff, is what appears as being optimized when
curve stands for 2| |2 (see Inequality (24)). Subplots (a), (b), and (c) are respectively replication-selection process is in action.
for 2-period orbits (xˆ(1) , xˆ(2) ) ≈ (0.12, 0.73 ), (0.36,0.89) and (0.14,0.80) of Leader We further remark that as a chaotic attractor has a dense set of
game (S = 7.5, T = 8.0). Subplot (d) depicts F1 vs.  for (xˆ(1) , xˆ(2) ) = (xˆ, xˆ) where
xˆ is mixed NE (xˆ ≈ 0.52).
countably infinite number of unstable periodic orbits, our study on
periodic orbits may potentially excite interest among researchers
to understand the meaning of chaos from the perspective of game
like, biology, economics, sociology, behavioural science, etc. Repli- theory. In particular, it would be an interesting problem to find out
cator dynamics is used as a model in problems involving social how to generalize the concept of ESS so as to connect it with the
dilemma (Iyer et al., 2014), molecular and cell biology (Hummert asymptotically stable nature of the chaotic attractor.
et al., 2014), economy (Friedman, 1998). The field of grammar We conclude by pointing out scope for extending the results
learning has been studied using replicator mutator as a dynami- reported in this paper. We remind ourselves that we have exclu-
cal model (Komarova et al., 2001; Nowak et al., 2001). Logit dy- sively worked with time-discrete dynamics in this paper. Thus, the
namics on the other hand, have mainly been applied in eco- extension of NE and ESS for periodic orbits in continuous replicator
nomical models (Fudenberg and Strzalecki, 2015; Lu, 2016) along dynamics remains an open problem. Furthermore, what happens if
with social and behavioural science (Ferraioli, 2013; Auletta et al., one relaxes the condition of infinite population is also quite an in-
2015). Brown–von Neumann–Nash dynamics has applications in teresting question. Specifically, it is a natural question to ask how
economic scenarios (Waters, 2009b) and evolution of heterogenous HSO or HSS can be defined for finite population in line with the
forecasting (Waters, 2009a). Projection dynamics was proposed as concept of ESS in finite population (Nowak, 2006).
a model in transport system (Nagurney and Zhang, 1997) and later
applied to complimentary formalism (Heemels et al., 20 0 0). Best
response dynamics have found applications in complex social net- CRediT authorship contribution statement
works (Fazli et al., 2018), internet and network economics (Nisan
et al., 2008). Clearly the vast applications of the aforementioned Archan Mukhopadhyay: Conceptualization, Methodology, For-
evolutionary dynamics in different domains are quite appealing mal analysis, Writing - original draft. Sagar Chakraborty: Concep-
and motivate one to study their dynamics in depth while appre- tualization, Methodology, Validation, Writing - original draft, Su-
ciating their implications. pervision.
It so happens that all the aforementioned dynamics show pe-
riodic and chaotic behaviours that are not merely the transient Acknowledgements
phases of the dynamics. We do not believe that it is justified
to attribute these robust non-trivial limit sets of phase trajec- The authors are grateful to Vimal Kumar and Varun Pandit for
tories to the inapplicability of the models just because the be- many insightful discussions on game theory.
haviours are not in direct conformity with the game theoretic con-
cept of NE. The emergence of chaos and periodic orbits in evolu-
tionary dynamics for two-player-two-strategy games simply indi- Appendix A. Two-player-n-strategy game
cates that the assumption of rationality may be unrealistic even in
the simplest setting. There are lack of compelling reasons behind For two-player-n-strategy game the dimension of payoff matrix
n (k )
how agents might have learned how to play NE (Kreps, 1990). In Π is n × n. The condition i=1 xi = 1 implies that the effective
a learning process, in a population of players meeting randomly dynamics is modelled by an (n − 1 )-dimensional dynamical sys-
and repeatedly, the players are endowed with some behavioural tem. The replicator map (refer Eq. (5)) has the following form for
rules of selecting strategies based on their experiences. Sato et al. n-strategy population game:
(2002) have illustrated that learning through a replicator model
even in an elementary setting of rock-paper-scissors games is prac-
n  
xi(k+1) = xi(k+1) + xi(k ) xh(k ) (Πx(k ) )i − (Πx(k ) )h , (27)
tically impossible because the resulting dynamics becomes chaotic.
h=1
In view of the above, in this paper, we argue that one needs h=i
to generalize the game theoretic concepts appropriately in order
to appreciate any non-fixed point behaviour of evolutionary game ∀i ∈ {1, 2, , n}. Let {xˆ (k) : xˆi(k) ∈ (0, 1 ), k = 1, 2, · · · , m} where
dynamics. To this end, we take an analytically tractable version of xˆ (i ) = xˆ ( j ) ∀i = j represent an m-periodic orbit of replicator map,
8 A. Mukhopadhyay and S. Chakraborty / Journal of Theoretical Biology 497 (2020) 110288

then in line with the derivation of Eq. (9) we arrive at φ2 (x(k) )/(1 − x(k) ) if and only if (Πx(k) )1 > (Πx(k) )2 . Hence, we
demand,
m n  
xˆi(k ) xˆh(k ) (Πxˆ (k ) )i − (Πxˆ (k ) )h = 0, (28)
φ1 ( x ( k ) ) φ2 ( x ( k ) )
k=1 h=1 − = β [(Πx(k ) )1 − (Πx(k ) )2 ], (32)
h=i x (k ) ( 1 − x (k ) )
∀i ∈ {1, 2, , n}. where β must be positive at all times. Now using condition 2 given
Now, in order to extend the definitions of HO and HSO to the above and Eq. (32), we can write the general form of monotone
general n-strategy games, we first need to appropriately define selection dynamics for two-player-two-strategy game as follows:
heterogeneity. Noting that even thought the population now has
β    
n types, the interactions are still confined to two-player interac- x(k+1) = x(k ) + Hx(k) Πx(k ) − Πx(k ) . (33)
tions. One can thus intuit that the heterogeneity must still be de- 2 1 2

fined pairwise. Consequently, for any arbitrary mixed state x(k) , we On comparing Eq. 33 with Eq. (6), it can easily be seen that one
define pairwise heterogeneity for any two pure types tagged by can still connect HO(m) and HSO(m) with the m-period orbit of
the indices, say, h and i where h = i and i, h ∈ {1, 2, , n}) as replicator map and its evolutionarily stability if we simply work
H ih(k) ≡ 2xi(k ) xh(k ) . Thus, every type has contribution in (n − 1 ) dif- with rescaled heterogeneity as Hx(k) → β Hx(k) .
x
ferent pairwise heterogeneities.
Definition of HO(m): The sequence of states {xˆ (k ) : xˆ(k ) ∈
References
(0, 1 ), k = 1, 2, · · · , m} where xˆ (i) = xˆ ( j ) ∀i = j, is an HO(m) if
∀i ∈ {1, 2, , n} and ∀j ∈ {1, 2, , m}, Alchian, A.A., 1950. Uncertainty, evolution, and economic theory. J. Polit. Econ. 58
(3), 211–221. doi:10.1086/256940.
m n m n
( j )T Auletta, V., Ferraioli, D., Pasquale, F., Penna, P., Persiano, G., 2015. Logit dynam-
Hxˆih(k) xˆ ih Πxˆ (k ) = Hxˆih(k) xTih Πxˆ (k ) , (29) ics with concurrent updates for local interaction games. Algorithmica 73 (36),
k=1 h=1 k=1 h=1 511–546.
h=i h=i Blume, L.E., 1993. The statistical mechanics of strategic interaction. Games Econ. Be-
hav. 5 (3), 387–424. doi:10.1006/game.1993.1023.
( j) ( j)
where xˆ ih (or xih ) is a mixed state that has same fraction Börgers, T., Sarin, R., 1997. Learning through reinforcement and replicator dynamics.
J. Econ. Theory 77 (1), 1–14. doi:10.1006/jeth.1997.2319.
of ith type as that of xˆ ( j ) (or x(j) ) and comprises only of ith Brown, G.W., von Neumann, J., 1950. Solutions of Games by Differential Equations,
( j)
and hth types; e.g., xˆ 13 = (xˆ1(k ) , 0, 1 − xˆ1(k ) , 0, · · · , 0 ) and xˆ 41
(k )
= (1 − 24. Princeton: Princeton University Press, pp. 73–79.
xˆ4(k ) , 0, 0, xˆ4(k ) , 0, · · · , 0 ).
Cressman, R., 2003. Evolutionary Dynamics and Extensive Form Games. MIT Press
Books, 1, first The MIT Press.
Definition of HSO(m): HSO(m) of a map—xi(k+1 ) = g(xi(k ) )— Cressman, R., Tao, Y., 2014. The replicator equation and other game dynamics. Proc.
Natl. Acad. Sci. 111 (Supplement 3), 10810–10817. doi:10.1073/pnas.1400823111.
is a sequence of states, {xˆ (k ) : xˆ(k ) ∈ (0, 1 ); k = 1, 2, · · · , m; xˆ (i ) = van Damme, E., 1991. Stability and Perfection of Nash Equilibria. Berlin: Springer
xˆ ( j ) ∀i = j} such that Berlin Heidelberg doi:10.1007/978- 3- 642- 58242- 4.
Darwin, C., 1871. The Descent of Man, and Selection in Relation to Sex, First London:
m n m n
John Murray.
( 1 )T ( 1 )T
Hxih(k) xˆ ih Πx(k ) > Hxih(k) xih Πx(k ) , (30) Darwin, C., 1887. The life and letters of Charles Darwin, including an autobiograph-
k=1 h=1 k=1 h=1 ical chapter, 1, Second London: John Murray.
h=i h=i Dawkins, R., 1989. The Selfish Gene. Oxford: Oxford University Press.
Fazli, M., Maazallahi, A., Habibi, J., Habibi, M., 2018. Controlling best response dy-
for any orbit {x(k ) : x(k ) ∈ (0, 1 ); k = 1, 2, · · · , m} of the map starting namics for network games. IEEE TNSE doi:10.1109/TNSE.2018.2814590.
(1 ) Feldman, M., Snappir, Y., Tamir, T., 2017. The efficiency of best-response dynamics.
in some infinitesimal neighbourhood B (1) \{xˆ } of xˆ (1 ) . In: Bilò, V., Flammini, M. (Eds.), Algorithmic Game Theory. Cham: Springer In-
xˆ
Without giving the tedious but straightforward details, we com- ternational Publishing, pp. 186–198.
ment that in line with the theorem proven for the case of 2- Ferraioli, D., 2013. Logit dynamics: A model for bounded rationality. SIGecom Exch.
12 (1), 34–37. doi:10.1145/2509013.2509018.
strategy games, following propositions hold true even for any gen- Friedman, D., 1998. On economic applications of evolutionary game theory. J. Evol.
eral n-strategy games: m-periodic orbit of replicator map is HO(m) Econ. 8 (1), 15–43. doi:10.10 07/s0 01910 050 054.
and vice versa; and locally asymptotically stable m-period orbit of Friedman, M., 1953. Essays in Positive Economics. Chicago: University of Chicago
Press.
the replicator map is HSO(m).
Fudenberg, D., Strzalecki, T., 2015. Dynamic logit with choice aversion. Econometrica
83 (2), 651–691. doi:10.3982/ECTA11846.
Gilboa, I., Matsui, A., 1991. Social stability and equilibrium. Econometrica 59 (3),
Appendix B. Case of monotone selection dynamics
859–867.
Harper, M., Fryer, D., 2015. Lyapunov Functions for Time-Scale Dynamics on Rieman-
Here we intend to show how the concepts of HO(m) and nian Geometries of the Simplex. Dyn. Games Appl. 5 (3), 318–333. doi:10.1007/
HSO(m) can be generalized for a rather broad class of dynamics, s13235- 014- 0124- 0.
Heemels, W., Schumacher, J., Weiland, S., 20 0 0. Projected dynamical systems in
called monotone selection dynamics (Cressman, 2003) which has a complementarity formalism. Oper. Res. Lett. 27 (2), 83–91. doi:10.1016/
replicator map as a special case. Specializing for the case of two- S0167-6377(0 0)0 0 042-0.
player-two-strategy games, a map, Hofbauer, J., Oechssler, J., Riedel, F., 2009. Brown–von neumann–nash dynamics: The
continuous strategy case. Games Econ. Behav. 65 (2), 406–429. doi:10.1016/j.geb.
xi (k+1) = xi (k ) + φi (x(k ) ), i = 1, 2, (31) 20 08.03.0 06.
Hofbauer, J., Sandholm, W.H., 2011. Survival of dominated strategies under evolu-
tionary dynamics. Theor. Econ 6 (3), 341–377. doi:10.3982/TE771.
models the selection dynamics, if the conditions given below are
Hofbauer, J., Schlag, K.H., 20 0 0. Sophisticated imitation in cyclic games. J. Evol. Econ.
satisfied: 10 (5), 523–543. doi:10.10 07/s0 01910 0 0 0 049.
Hofbauer, J., Sigmund, K., 2003. Evolutionary game dynamics. Bull. Am. Math. Soc.
1. The simplex, σ 2 , is a forward invariant of this map. 40 (4), 479–519. doi:10.1090/S0273- 0979- 03- 00988- 1.
2. φ1 (x(k ) ) + φ2 (x(k ) ) = 0 for all non-negative integer k. Hummert, S., Bohl, K., Basanta, D., Deutsch, A., Werner, S., Theißen, G., Schroeter, A.,
Schuster, S., 2014. Evolutionary game theory: cells as players. Mol. BioSyst. 10,
3. Both φ 1 (x(k) ) and φ 2 (x(k) ) are Lipschitz continuous on some 3044–3065. doi:10.1039/C3MB70602H.
open neighbourhood in the simplex. Iyer, S., Reyes, J., Killingback, T., 2014. An application of evolutionary game theory to
4. φ 1 (x(k) )/x(k) and φ2 (x(k ) )/(1 − x(k ) ) are continuous real valued social dilemmas: The traveler’s dilemma and the minimum effort coordination
game. PLoS One 9 (4), 1–11. doi:10.1371/journal.pone.0093988.
functions on the simplex. Komarova, N.L., Niyogi, P., Nowak, M.A., 2001. The evolutionary dynamics of gram-
mar acquisition. J. Theor. Biol. 209 (1), 43–59. doi:10.10 06/jtbi.20 0 0.2240.
Now to ensure that the dynamics is a monotone selection dy- Kreps, D.M., 1990. Game Theory and Economic Modelling. Oxford: Oxford University
namics we impose the condition of monotonicity: φ1 (x(k ) )/x(k ) > Press.
 A. Mukhopadhyay and S. Chakraborty / Journal of Theoretical Biology 497 (2020) 110288 9

Lahkar, R., Sandholm, W.H., 2008. The projection dynamic and the geometry of pop- Samuelson, L., 2002. Evolution and game theory. J. Econ. Perspect. 16 (2), 47–66.
ulation games. Games Econ. Behav. 64 (2), 565–590. doi:10.1016/j.geb.2008.02. doi:10.1257/0895330027256.
002. Special Issue in Honor of Michael B. Maschler Sandholm, W.H., Dokumacäś, E., Lahkar, R., 2008. The projection dynamic and the
Lu, J., 2016. Random choice and private information. Econometrica 84 (6), 1983– replicator dynamic. Games Econ. Behav. 64 (2), 666–683. doi:10.1016/j.geb.2008.
2027. doi:10.3982/ECTA12821. 02.003. Special Issue in Honor of Michael B. Maschler
Maynard Smith, J., 1972. Game theory and the evolution of fighting. Edinburgh: Ed- Sato, Y., Akiyama, E., Farmer, J.D., 2002. Chaos in learning a simple two-person
inburgh University Press. game. Proc. Natl. Acad. Sci. 99 (7), 4748–4751. doi:10.1073/pnas.032086299.
Maynard Smith, J., Price, G.R., 1973. The logic of animal conflict. Nature 246 (5427), Selten, R., 1975. Reexamination of the perfectness concept for equilibrium points
15–18. doi:10.1038/246015a0. in extensive games. Internat. J. Game Theory 4 (1), 25–55. doi:10.1007/
Myerson, R.B., 1978. Refinements of the nash equilibrium concept. Internat. J. Game BF01766400.
Theory 7 (2), 73–80. doi:10.1007/BF01753236. Skyrms, B., 1992. Chaos and the explanatory significance of equilibrium: Strange at-
Nagurney, A., Zhang, D., 1997. Projected dynamical systems in the formulation, tractors in evolutionary game dynamics. PSA: Proceedings of the Biennial Meet-
stability analysis, and computation of fixed-demand traffic network equilibria. ing of the Philosophy of Science Association 1992 (2), 374–394. doi:10.1086/
Transp. Sci. 31 (2), 147–158. doi:10.1287/trsc.31.2.147. psaprocbienmeetp.1992.2.192851.
Nelson, R.R., Winter, S.G., 2002. Evolutionary theorizing in economics. J. Econ. Per- Smith, M.J., 1984. The stability of a dynamic model of traffic assignment–an appli-
spect. 16 (2), 23–46. doi:10.1257/0895330027247. cation of a method of lyapunov. Transp. Sci. 18 (3), 245–252.
Nisan, N., Schapira, M., Zohar, A., 2008. Asynchronous best-reply dynamics. In: Pa- Taylor, P.D., Jonker, L.B., 1978. Evolutionary stable strategies and game dynamics.
padimitriou, C., Zhang, S. (Eds.), Internet and Network Economics. Berlin, Hei- Math. Biosci. 40 (1), 145–156. doi:10.1016/0 025-5564(78)90 077-9.
delberg: Springer-Verlag, pp. 531–538. Vilone, D., Robledo, A., Sánchez, A., 2011. Chaos and Unpredictability in Evolution-
Nowak, M.A., 2006. Evolutionary Dynamics: Exploring the Equations of Life. Harvard ary Dynamics in Discrete Time. Phys. Rev. Lett. 107 (3), 038101. doi:10.1103/
University Press. PhysRevLett.107.038101.
Nowak, M.A., Kamarova, N.L., Niyogi, P., 2001. Evolution of universal grammar. Sci- Waters, G., 2009. Modeling heterogeneous forecasting strategies the right way. Un-
ence 291 (5501), 114–118. doi:10.1126/science.291.5501.114. published manuscript.
Pandit, V., Mukhopadhyay, A., Chakraborty, S., 2018. Weight of fitness deviation Waters, G.A., 2009. Chaos in the cobweb model with a new learning dynamic. J.
governs strict physical chaos in replicator dynamics. Chaos 28 (3), 033104. Econ. Dyn. Control 33 (6), 1201–1216. doi:10.1016/j.jedc.20 08.11.0 04.
doi:10.1063/1.5011955.
Rice, S.H., 1961. Evolutionary Theory: Mathematical and Conceptual Foundations,
First Massachusetts: Sinauer Associates, Inc..