Ordering structured populations in multiplayer cooperation games

Spatial structure greatly affects the evolution of cooperation. While in two-player games the condition for cooperation to evolve depends on a single structure coefficient, in multiplayer games the condition might depend on several structure coefficients, making it difficult to compare different population structures. We propose a solution to this issue by introducing two simple ways of ordering population structures: the containment order and the volume order. If population structure is greater than population structure in the containment or the volume order, then can be considered a stronger promoter of cooperation. We provide conditions for establishing the containment order, give general results on the volume order, and illustrate our theory by comparing different models of spatial games and associated update rules. Our results hold for a large class of population structures and can be easily applied to specific cases once the structure coefficients have been calculated or estimated.


Polytopes
A (convex) polyhedron can be defined as the intersection of finitely many closed halfspaces in R n , i.e., as the set of solutions to a system of m linear inequalities where T is a real m × n matrix, and c a real vector of size m. A (convex) polytope is a bounded polyhedron. When given by a system of linear inequalities such as (1), a polytope is said to be given in its H-representation (18).
We consider symmetric games with two pure strategies (A and B) between d players. A focal player's payoff depends on its own strategy and on its d − 1 co-players'. If j co-players play A, a focal player obtains a j if it plays A or b j if it plays B. We focus on a subset of games that we call "cooperation games". In these games A represents cooperation; B, defection; and payoffs are such that: (i) irrespective of its own strategy, a focal player prefers co-players to cooperate, and (ii) mutual cooperation is favored over mutual defection. In terms of the payoffs of the game, these conditions respectively imply a j+1 ≥ a j and b j+1 ≥ b j for j = 0, 1, . . . , d − 2, and We further restrict payoffs to values between 0 and 1, so that 0 ≤ a j ≤ 1 and 0 ≤ b j ≤ 1 for j = 0, 1, . . . , d − 1.
The previous inequalities give the H-representation of a polytope that we denote by P.
For weak selection, strategy A is favored over B if (17) where σ 0 , . . . , σ d−1 are the d structure coefficients of the population structure (and associated update rule) under consideration. For a given population structure S i with vector of structure coefficients σ i , inequalities (2), (3), and (4) together with the selection condition (5) give the Hrepresentation of a polytope that we denote by Q i .

Structure coefficients
The structure coefficients σ of a given model of population structure can be calculated from the condition ρ A > ρ B , where ρ X denotes the fixation probability of a single mutant playing X (either A or B) in a population of residents playing the opposite strategy (9). This condition can be written in terms of selection coefficients (dependent on the payoffs of the game and the demographic parameters of the model) and expected coalescence times under neutrality (12). However, the expected coalescence times required for calculating the structure coefficients of general d-player games can be difficult to obtain (5,12). At least for simple population structures, the condition ρ A > ρ B can be more easily calculated from first principles, and the structure coefficients extracted from the resulting expressions. This is the approach we followed here.

Moran process on a well-mixed population
The condition ρ A > ρ B is given by (ref. (2), Eq. S19) which can be rewritten as Dividing both sides of the inequality by N and comparing with the selection condition (5), we obtain

Aspiration dynamics on a well-mixed population
The condition ρ A > ρ B is given by Due to the symmetry of binomial coefficients d−1 j = d−1 d−1−j , this can be rewritten as Comparing this last expression with the selection condition (5), we obtain

death-Birth process on a cycle
Let us consider the model of population structure discussed in ref. (14). Each individual is placed on the node of a cycle. Every sequence of d players defines the participants in a d-player game. Individuals accumulate the payoffs from the d games they are involved in, each with d players. These payoffs are transformed to "fitness" via an exponential payoff-to-fitness mapping (11). Each time step, a randomly chosen individual is selected to die and its two neighbours compete for the vacant spot with a probability proportional to fitness. If the population starts with a single mutant, mutants form a single connected cluster in the cycle at any time. The state of the population can hence be captured by the number of A-players in this cluster, i. Denote by f 0 A (i) (f 0 B (i)) the payoff of an A-player (B-player) lying immediately at the boundary of a cluster of B-players (A-players), and by f 1 A (i) (f 1 B (i)) the payoff of an Aplayer (B-player) right next to it (Fig. 1). Let T ± i be the probability that the number of A-players increases (+) or decreases (−) in one time step, when there are i A-players in the population.
where ω is a parameter measuring the strength of selection. Strategy A is favored if ρ A > ρ B , which is equivalent to (6) For weak selection, condition (7) is equivalent to Replacing (6) into (8), we get d dω The above expression is a linear combination of the payoff entries a j and b j . Further, by the selection condition (5), the coefficients of a j are the same as those of b d−1−j . Thus it is only necessary to calculate the coefficients of a j , which only depend on the first two terms of (9).
Using the expressions for f 0 A (i) and f 1 A (i) (see ref. (14), Appendix B) we finally obtain: It is noteworthy that the expression is the payoff of an A-player (resp. B-player) next to the boundary between A-players and B-players. f 1 is the payoff of an A-player (resp. B-player) second-to-next to the boundary between A-players and B-players. obtained in ref. (14) is a sufficient condition for strategy A to be more abundant than strategy B. Our result (5) with structure coefficients given by (10) is the necessary and sufficient condition. In addition, we note that our result holds for general payoff-to-fitness mapping f , provided that f (0) is non-vanishing (16). Finally, we note that the stricture coefficients given in (10)

Moran process on a group splitting model
Consider the following multiplayer extension (4) of the group splitting model of ref. (10). The population is subdivided into m groups. Each population is allowed to grow to its maximum size n, then splits with probability q. Within populations, random groups of d individuals form and interact in a d-player game. When group splitting is rare (q 1) and the mapping between payoffs and fitness is given by an exponential function, the ratio of fixation probabilities is given by (ref. (4), Eq. 15): where are, respectively, the "direct" and "indirect" gains from switching from strategy A to strategy B (see ref. (7), Eqs. 6 and 7), and can be interpreted as the "scaled relatedness coefficient" of this model when the migration rate is zero (see ref. (13), Eq. B.4). From (11), a necessary and sufficient condition for ρ A > ρ B is that: which can be rearranged in terms of payoffs and structure coefficients in the form of the left hand side of (5): Hence, condition (13) can be written as: After dividing both sides of the inequality by 1 − κ, inserting the value of κ given in (12), and comparing with (5), we obtain

Normalized structure coefficients
For all population structures discussed in this section and listed in Table 1 of the main text, the structure coefficients are nonnegative. This is also true for many other population structures, at least for d = 2 (9). In these cases, the structure coefficients can be normalized so that the containment order can be investigated using stochastic orders (8). Henceforth, we refer to the normalized structure coefficients by σ = (σ 0 , . . . , σ d−1 ). Table 1 lists the normalized structure coefficients for the examples of population structures previously discussed.

Containment order 3.1 A sufficient condition leading to the containment order
Consider two population structures S 1 and S 2 characterized by the (normalized) structure coefficients σ 1 and σ 2 , and associated random variables J 1 and J 2 , respectively. Let us also define the Model Normalized structure coefficients Moran process on a well-mixed population Moran process on a group splitting model Table 1: Normalized structure coefficients for multiplayer evolutionary game models. sequence of "gains from flipping": i.e., the gains in payoff experienced by a focal B-player interacting with j A-players (and d − 1 − j B-players) after all players in the group, including the focal, flip their strategies (A-players become B-players and vice versa). Because of condition (2), the gains from flipping are increasing. Hence, a sufficient condition for S 1 to be greater than S 2 in the containment order is that A sufficient condition for this is that which is fulfilled by definition if J 1 is greater than J 2 in the (usual) stochastic order, denoted by There are many conditions leading to the stochastic ordering of two random variables (ref.
where ς is the distribution function corresponding to σ, i.e., A simple sufficient condition leading to the set of inequalities given by (17) and hence to J 1 ≥ st J 2 is that S − (σ 1 − σ 2 ) = 1 (where S − (a) is the number of sign changes of the sequence a) and the sign sequence is −, + (ref. (8), p. 10). That is, if the structure coefficients σ 1 "put more weight" in larger values of j than the structure coefficients σ 2 , then J 1 ≥ st J 2 . We summarize this observation in the following proposition.

A sufficient condition leading to the incomparability in the containment order
Given two population structures S 1 and S 2 , it could be that neither S 1 ≥ con S 2 nor S 1 ≤ con S 2 holds true. We are also interested in establishing a simple sufficient condition leading to such incomparability in the containment order, that we denote by S 1 con S 2 . In order to derive this, suppose that the structure coefficients of S 1 and S 2 cross each other twice, i.e., that Condition (19) implies that S − (ς 1 − ς 2 ) = 1 (ref. (15), p. 621) and hence that (17) does not hold true. This in turn implies that J 1 and J 2 are incomparable in the stochastic order, i.e., J 1 st J 2 .
Showing that (19) also implies S 1 con S 2 however requires some additional arguments. Indeed, note that J 1 st J 2 does not necessarily imply S 1 con S 2 : the stochastic order is a sufficient but not a necessary condition leading to the containment order (cf. (15) and (16)).
In order to prove that (19) leads to S 1 con S 2 , we make use of two other stochastic orders: the increasing convex order and the increasing concave order (ref. (8), p. 181). A random variable J 1 is said to be greater than J 2 in the increasing convex (resp. concave) order, denoted by , for all increasing convex (resp. concave) functions φ : R → R. (20) A simple condition leading to these orders is given in the following lemma.
Lemma 1 (A sufficient condition leading to the increasing convex (resp. concave) order). Let X and Y be two random variables with density functions p and q respectively. If Proof. Denote by P and Q the distribution functions associated to X and Y , respectively. Condition (21) implies (ref. (15), p. 621) S − (P − Q) = 1 with sign sequence +, − (resp. −, +).
Let us now consider two population structures S 1 and S 2 whose structure coefficients satisfy (19). Without loss of generality, suppose that the sign pattern is +, −, +. By Lemma 1, it follows that J 1 ≥ icx J 2 and J 2 ≥ icv J 1 . This suggests that there might be both (i) games with increasing and convex gains from flipping f j for which S 1 (but not S 2 ) fulfills the selection condition (5), and (ii) games with increasing and concave gains from flipping f j for which S 2 (but not S 1 ) fulfills the selection condition (5).
As an example of such games, consider a club goods game between cooperators (A) and defectors (B), where cooperators pay a cost c > 0 in order to provide an excludable collective good that only cooperators can use, while defectors refrain from contributing and hence from using the good (7). This game is characterized by the payoff sequences: where v k gives the value of the collective good as a function of the total number of cooperators, k = j + 1, and c is the payoff defectors obtain. We further assume that v k is given by where v > 0 is some baseline value, and u > 0 is a synergy or discounting parameter (3). Furthermore, we require that v/c > γ, where 1/γ = (1 − u d )/(1 − u), so that (3) is fulfilled. The gains from flipping of this game are then given by Let us first impose the condition: so that S 1 but not S 2 satisfies the selection condition (5). Condition (23) is satisfied if Note that (24) is satisfied if u > 1, because in this case (1 − u j+1 )/(1 − u) is increasing and convex and J 1 ≥ icx J 2 . Additionally, 1/γ ≤ 1/γ 1 always holds.
Let us now impose the condition: so that S 2 but not S 1 satisfies the selection condition (5). In this case, (25) is satisfied if which holds true if 0 < u < 1, as in this case (1 − u j+1 )/(1 − u) is increasing and concave and J 2 ≥ icv J 1 . Additionally, 1/γ ≤ 1/γ 2 always holds. We summarize the previous observations in the following proposition.

Proposition 2 (A sufficient condition leading to the incomparability in the containment order).
Let S 1 and S 2 be two population structures with (normalized) structure coefficients σ 1 and σ 2 , respectively. If S − (σ 1 − σ 2 ) = 2 then S 1 con S 2 , i.e., neither S 1 ≥ con S 2 nor S 1 ≤ con S 2 hold true. Moreover, if the sign sequence of σ 1 − σ 2 is +, −, + (resp. −, +, −) then it is possible to find cooperation games with convex (resp. concave) gains from flipping f j such that the selection condition (5) is satisfied for S 1 but not for S 2 (resp. for S 2 but not for S 1 ) and cooperation games with concave (resp. convex) gains from flipping f j such that the selection condition (5) is satisfied for S 2 but not for S 1 (resp. for S 1 but not for S 2 ).

The containment order is a total order for d = 2 but a partial order for d ≥ 3
Propositions 1 and 2 allow us to prove the following result.
Proposition 3 (The containment order is total for d = 2 but partial for d = 3). Consider the set of all possible population structures {S} for a given group size d. {S} is totally ordered under ≤ con for d = 2 but only partially ordered under ≤ con for d ≥ 3.
Proof. For d = 2, the probability mass function given by the normalized structure coefficients σ consists of only two points. Consequently, σ 1 − σ 2 has either (i) no sign changes (i.e., σ 1 = σ 2 ), which implies S 1 = con S 2 ; (ii) a sign change from − to +, which implies S 1 ≥ con S 2 ; or (iii) a sign change from + to −, which implies S 1 ≤ con S 2 . For d ≥ 3, the probability mass function given by the normalized structure coefficients σ consists of d ≥ 3 points. In this case, it is always possible to find S 1 and S 2 such that σ 1 − σ 2 has two sign changes. In this case, neither S 1 ≥ con S 2 nor S 1 ≤ con S 2 hold true.

Examples
In the following, we state several results concerning the containment order for the population structures listed in Table 1. We omit the proofs, as they are straightforward applications of Propositions 1 and 2 above.  be well-mixed populations of size N ≥ d, updated with aspiration dynamics and a Moran process, respectively. We have:

Volume order
In the following, we give a formula for the volume of cooperation games, as defined by inequalities (2)- (4). For this, we find convenient to define I as the polytope given by inequalities (2) and (4), and J as the polytope given by inequalities (2), (4), and a d−1 ≤ b 0 (which is the opposite of (3)). The volumes of these two polytopes are easy to calculate exactly using probabilistic arguments. Indeed, we have the following two lemmas. Lemma 2 (Volume of I). We have that Proof. Calculating the volume of I is equivalent to calculating the probability that two sequences (a 0 , . . . , a d−1 ) and (b 0 , . . . , b d−1 ), with elements randomly and independently drawn from the in- For each sequence, the probability of having a randomly ordered sequence of length d is 1/d!, since d! is the number of permutations of d distinct objects and only one of such permutations will be given in the specified order. Since the two sequences are independent, the total probability is given by 1/(d!) 2 .
Lemma 3 (Volume of J ). We have that Proof. Calculating the volume of J is equivalent to calculating the probability that the sequence with elements randomly and independently drawn from the interval Following the same argument as in the proof of Lemma 2, this probability is equal to 1/(2d)!.
Making use of these two lemmas, we can find an expression for the volume of P, the polytope of cooperation games. We state this result in the following proposition.
Proposition 10 (Volume of cooperation games). The volume of cooperation games is given by Proof. Follows from Lemmas 2 and 3 upon noticing that P = I − J .
Some population structures, such as large well-mixed populations updated with a Moran process, and finite well-mixed populations updated with the aspiration dynamics, are such that their structure coefficients are symmetric, i.e., σ j = σ d−1−j for all j. We are intereted in calculating the fraction of cooperation games for which such population structures favor strategy A. In order to calculate this result, we need the following lemma. The lemma may appear to be obvious for symmetry reasons, but the additional requirement that we are dealing with cooperation games (and hence a subset of the hypercube of all possible games) adds a further complication. Lemma 4. Let S be a population structure with symmetric structure coefficients, i.e., σ j = σ d−1−j for all j. Let also I + be the subset of I for which A is favored over B. Then Proof. Denote by I − and I 0 the subsets of I such that j=0 σ j (a j − b d−1−j ) = 0, respectively. Then we arrive at a partition of set I, namely I = I + ∪ I 0 ∪ I − . Since there must exist a j * such that σ j * = 0, the solution space of d−1 j=0 σ j (a j − b d−1−j ) = 0 is of lower dimension than 2d, and hence Vol(I 0 ) = 0. Thus Vol(I) = Vol(I + ) + Vol(I − ).
Proposition 11. Let S be a population structure with positive symmetric structure coefficients, i.e., σ j = σ d−1−j for all j, and Q the polytope asociated to all cooperation games for which A is favored over B under S. Then Vol(Q) Vol(P) = 1 2 which is a decreasing function of d, and is equal to 1/2 in the limit of large d.
Using Lemmas 2 and 3, we finish the proof.