Biological markets in cooperative breeders: quantifying outside options

A major aim in evolutionary biology is to understand altruistic help and reproductive partitioning in cooperative societies, where subordinate helpers forego reproduction to rear dominant breeders' offspring. Traditional models of cooperation in these societies typically make a key assumption: that the only alternative to staying and helping is solitary breeding, an often unfeasible task. Using large-scale field experiments on paper wasps (Polistes dominula), we show that individuals have high-quality alternative nesting options available that offer fitness payoffs just as high as their actual chosen options, far exceeding payoffs from solitary breeding. Furthermore, joiners could not easily be replaced if they were removed experimentally, suggesting that it may be costly for dominants to reject them. Our results have implications for expected payoff distributions for cooperating individuals, and suggest that biological market theory, which incorporates partner choice and competition for partners, is necessary to understand helping behaviour in societies like that of P. dominula. Traditional models are likely to overestimate the incentive to stay and help, and therefore the amount of help provided, and may underestimate the size of reproductive concession required to retain subordinates. These findings are relevant for a wide range of cooperative breeders where there is dispersal between social groups.


Introduction
Altruistic helping behaviour occurs throughout the animal kingdom despite costs to helpers' direct fitness. In cooperatively breeding animals, subordinates care for, defend and provision the offspring of dominant breeders, while foregoing or delaying their own reproduction [1,2]. A range of factors has been identified to explain the evolution and maintenance of this phenomenon, including both direct fitness benefits, such as inheritance of the breeding position [3,4], and indirect fitness benefits obtained through helping a relative [5,6]. However, there is an increasing awareness in the literature of the limitations of traditional theoretical models, and a call for more complex models that more realistically describe the social environment of individuals [7][8][9][10][11][12]. Specifically, traditional models predicting the level of help and reproductive skew in cooperative breeders often make a key assumption: that a subordinate helper's only alternative to staying and helping in its current group is to leave and breed solitarily [13][14][15][16][17][18]. However, breeding alone is often unfeasible or highly risky [3,19,20], leading to the prediction that subordinates should accept a high workload and a small share or zero part of the reproduction, in order to remain in the group.
Recent literature increasingly suggests that in order to correctly estimate the costs and benefits associated with staying and helping in a group, one must compare the payoffs of that decision with an individual's true alternative options [7,8,11,21]. In reality, a subordinate's alternative options may include switching to another group or recruiting other cooperative partners to initiate a new breeding group [11,20,22,23]. If such alternative options could lead to higher fitness payoffs than solitary breeding, payoff distributions may have been miscalculated in past studies, overestimating the incentive for subordinates to stay and help. Hence, future studies are encouraged to include the following: partner choice rather than partner control, where sanctioning of uncooperative partners is replaced by partner switching [12,23 -25]; outside options beyond solitary breeding [9,10]; asymmetric relationships where the exchange of behaviours is more valuable for one of the parties [21,26,27]; and n-player interactions not achievable in traditional 2-player cooperative games [21,28]. These modifications can be achieved by invoking biological market models [21]. Biological market theory predicts that competition for cooperative partners will affect the value of commodities exchanged between individuals of different trader classes [9,10]. In cooperative breeders, subordinates may be seen as effectively exchanging helping behaviour for group membership [15,29,30], and the value of helping behaviour may therefore be affected by the supply of and demand for help in the market. Subordinates may be described as 'paying to stay' [30][31][32] or dominants as 'paying for help' [33,34], depending on which commodity is in focus. For example, when there is competition among dominants for a limited supply of helpers so that help is in high demand, dominants may be willing to accept subordinates paying less for group membership through reducing their work efforts. Similarly, dominants might be willing to pay more for help by granting a higher share of reproduction to subordinates [9,10,22,33,34].
Several studies of cooperatively breeding mammals [31,33,34], fishes [32,35,36], birds [30,37,38] and insects [22] have found support for the concept that dominants and subordinates exchange commodities as described above. In the cooperatively breeding paper wasp, Polistes dominula, we previously reported data consistent with the hypothesis that dominants have to accept a lower payment from their subordinates when competition for help is increased in the population [22]. We first showed that wasps had outside options and a choice of cooperative partners. We then experimentally increased the amount of outside options available to subordinates and found that subordinates, as a result, decreased their work efforts [22]. These results suggest that there is a biological market in this species where the supply of outside options affects the exchange of cooperative behaviours within groups. However, in order to wholly understand the dynamics between dominants and subordinates, we need to know not only the number of alternative options available but also their quality. Only by evaluating the attractiveness of outside options will the behavioural decisions of cooperative partners be clear. Here, we quantify the outside options available to P. dominula wasps and estimate the fitness payoffs associated with these options. We further evaluate how partner choice may affect the payoff distribution between cooperative partners, and assess the implications this may have for cooperative theory.
The nesting behaviour and social organization of P. dominula is well studied, and our study sites offer large samples of small groups [8,22,39]. At these sites, thousands of mated females from the same generation emerge simultaneously from hibernation in early spring and found hundreds of nests along cactus hedges (Opuntia ficus-indica). Groups of typically fewer than 10 females and small numbers (approx. 6.4% of all females in [40]) of solitary breeders rear workers that mature during late spring and early summer. Here we focus on the pre-worker stage where groups of similar-aged females live as cooperative breeders. The dominant breeder lays all or most of the eggs, while subordinates build and expand the nest, forage and help care for the offspring of the dominant [41]. Nest residents often consist of genetically related individuals (sisters and cousins), but a significant proportion of subordinates are unrelated to the dominant they are helping [42 -44]. The chance of inheriting the breeding position or obtaining a small share of the reproduction has been used to explain the presence of unrelated helpers in this species: Leadbeater et al. [3] found that the amount of direct fitness obtained as a subordinate was greater than through solitary breeding [3]. However, if helpers have alternative options available that offer higher fitness payoffs than solitary breeding, the incentive to stay and help in their current groups may previously have been overestimated.
We ask the following questions. (i) Do available nesting options include high-payoff alternatives? Alternative options will affect the predictions of existing models only if they offer a higher payoff than solitary nesting. (ii) Do alternative options differ from observed choices in ways that should affect direct and indirect fitness, such as inheritance rank and relatedness to the dominant? We predict that alternative options are inferior to observed choices: in a biological market, individuals are expected to assess their options and make the choice that offers the highest payoff [10]. (iii) Is it costly for dominants to reject an additional cooperative partner? We expect help to be in high demand because productivity and group survival increase with the number of helpers in P. dominula [3,22], so we predict that rejecting a joiner represents a cost to dominant breeders.

Methods
(a) Study species, field site and handling of animals Polistes dominula is a primitively eusocial (cooperatively-breeding) wasp lacking morphological castes. At our field site, females from the same generation found nests in early spring after overwintering. The first female offspring to mature in late spring become workers and those maturing during summer mate and overwinter, to restart the cycle next spring [3].
Experiments were carried out in a rural area in southern Spain, close to Conil de la Frontera, Cadiz (N 36817 0 10.9 00 W 6803 0 57.8 00 ) [3,22] during two field seasons: March -May 2013 and 2014. We tagged and numbered a total of approximately 700 nests: approximately 475 nests in two sub-populations in 2013 and approximately 225 nests in one subpopulation in 2014 (figure 1; same data as used for 'the partner choice experiment' in [22]). We further recorded the location of all nests along three axes (to nearest 5 cm), allowing us to calculate the distances between nests in a 3D space.
Combining the two field seasons, individuals from approximately 200 of these nests were collected during early mornings, before sunrise (6.00-7.00). In the laboratory we gave each wasp a unique code of four coloured dots on her thorax using enamel paints; measured the length of one of her wings to the nearest 0.1 mm; and obtained a DNA sample by cutting the tarsus from a middle leg. Tarsus samples were kept in 100% ethanol at approximately 48C until used for genotyping. Wasps were released close to their nests the same morning before 11.00. When wasps were permanently removed as a part of an experimental treatment they were either freeze-killed or released at a field site 2.5 km away: none returned to her original site. The day after residents were marked on a nest, we checked the nest in the evening for additional, unmarked residents. Any unmarked residents were collected the following morning and marked as described above. Once all nest residents were marked, we started daytime and evening censuses. Daytime censuses consisted of 3 -4 spot-checks per day (min. 30 min between each census) on sunny days every 2 -4 days, where the presence or absence of nest residents were recorded during the main foraging period (11.00-17.00). From the daytime censuses we identified the social rank of each resident in the linear dominance hierarchy: the dominant breeder spends the most time on the nest, while the lowest-ranked individual spends the most time away from the nest foraging [45].
In the evenings, nest residents return to their nests for the night. During evening censuses (18.00 -20.00), performed every 2 -4 days, we searched focal nests for new joiners. To mark a new joiner with minimal disturbance, we carefully applied a single pink paint dot to its abdomen while it was on the nest (day 0 of the joiner experiment). We videoed a subset of 21 focal nests on day 1 for three hours during the main foraging period (11.00-17.00). The following morning (day 2), we caught and marked the joiner as described above. On day 3 we confirmed the presence of the joiner during an evening census, so that we could plan to apply treatment the following morning (day 4; treatment morning). If the joiner was not present on its nest during one of these checks, we looked for it on the nest for a maximum of three days. If the joiner re-appeared within this period we continued with the next step of the procedure; if it did not we resumed normal censuses of the nest.
On the morning of treatment in the joiner experiment, we applied one of three treatments (n treated focal nests ¼ 62).
(1) Joiner's first choice: this was our control treatment where the joiner was allowed to stay and no nest residents were removed.
(2) Joiner's second choice: we removed the joiner's first nest choice by permanently removing the nest and all of its residents, while immediately releasing the joiner itself. If any established residents were absent from the nest, we left the nest in situ for a maximum of 48 hours before removing it, allowing us to attract and remove remaining residents. (3) Joiner removal: we permanently removed the joiner (or both joiners if two had joined) while releasing all other residents near to the nest.
In addition to applying one of the three described treatments, we also recorded the presence of all residents on focal nests by collecting all wasps on their nest, recording their IDs and releasing individuals immediately according to treatment. We further performed a brood census on each focal nest, which included counting the number of cells and categorizing the development of brood within each cell. Nest-level brood values were later summed as follows: small larva (given a value of 1.5), medium larva (2), large larva (3) and pupa (4); a cell without a larva or pupa was assumed to contain an egg (1).
In addition to the joiner experiment, we carried out a subordinate experiment similar to treatment 2 ( joiner's second choice), but using established low-ranking subordinates, rather than new joiners, from a separate set of nests. In each of 34 nests that had not received joiners during our observations, we chose one of the lowest ranking subordinates and released it after removing the nest and the remaining nest residents, as in the joiner experiment, treatment 2.
Following the treatments, we searched for released joiners and subordinates in all nests in the sub-populations during daytime and evening censuses every 2 -4 days. When a released individual was found on a new host nest with unmarked residents, we waited 2 -3 days and then collected and marked the residents. We also resumed daytime censuses on all focal nests (including these new host nests) 2 -3 days after treatment, and performed brood census as described above every 10 -15 days. When the first worker(s) matured on a nest, we performed one final brood census, and then discontinued all censuses on that nest.

(c) Video analysis
Each video was watched by one of seven people who recorded when nest residents left and returned to the nest, and all behavioural interactions. Observers were all trained by one person, who spot-checked for consistency. Behavioural interactions were ranked according to level of aggressiveness: antennation (given a value of 1), food sharing (2), and aggression (3; including all more aggressive encounters such as bite, chew and lunge). Two aggression values were calculated for each individual: the sum of values for all behaviours initiated by that individual, and the sum of values for all received behaviours during the full video recording.
Foraging returns brought back to the nest were ranked according to value in the following way: nothing visible (given a value of 0), nesting material (1), liquid food, as evidenced by trophallaxing (2), or a solid food ball (3). Foraging return values were calculated for each individual as the sum of values during the full video recording.

(d) Genotyping and relatedness
Protocols were identical to those described previously [22]. Briefly, DNA was extracted from tarsus samples and samples were genotyped at nine microsatellite loci used previously in studies of the same population [3,22,44,46,47]. Each locus had between 6 and 51 different alleles in our samples (median in 2013 ¼ 13; median in 2014 ¼ 11). All loci were amplified in a single multiplex reaction using the Qiagen multiplex PCR kit (Qiagen, Venlo, The Netherlands). RELATEDNESS v. 5.0.8 software [48] was used to calculate relatedness between joiners and nest residents as in [22]. The Full Sibship Reconstruction procedure in KINGROUP v. 2 software [44,49] was used to identify groups of sisters among the nests in each block (primary hypothesis: haplodiploid sisters; null hypothesis: haplodiploid cousins) [3]. We then counted the number of sisters each resident had in its own nest and in other nests. Only individuals with at least six out of nine loci scored successfully were used (median number of successful loci per sample ¼ 9); 1996 out of 2011 wasps were successfully genotyped.

(e) Statistics
All statistical analyses were performed using the statistical software R [50]. Whenever appropriate, non-parametric tests were used, and whenever the effect of more than one predictor was tested, GLMs (generalized linear models) or GLMMs (generalized linear mixed models) were used [51]. For count data we used Poisson error and tested for overdispersion: negative binomial error was used if models were overdispersed, and again we tested to ensure these models were no longer overdispersed before proceeding. For models with continuous data we used a Gaussian error structure and checked to ensure that residuals were homogeneous and normally distributed. Non-significant predictor variables ( p . 0.05) were removed from full models in order to obtain more reliable p-values for the remaining predictors. When analysing data from video recordings, we incorporated nest ID and the ID of the person watching the video as random effects. When analysing aggression and foraging return values we used the glmmADMB package [52] to build GLMMs with negative binomial error. This package further allowed us to account for zero inflation in the aggression models.

(b) Consequences of rejecting a joiner for established nest residents
We removed one or two joiners from each of 16 joinerremoval nests (21 joiners removed). After treatment, more joiner-removal nests received extra joiners (7 out of 16 nests: 43.8%) than did control nests where joiners were allowed to stay (2 out of 14 nests: 14.3%). However, the difference in number of extra joiners received in the two treatments was not significant (figure 3a; Mann -Whitney U test: , and the extra joiners received were not enough to replace those removed: control nests received significantly more joiners overall (including focal joiners) than joiner-removal nests did excluding removed focal joiners (figure 3b; Mann -Whitney U test: W ¼ 169, Original dominants were no more likely to lose their dominant breeding positions in control nests where joiners were allowed to stay than in joiner-removal nests. After treatment, the dominant lost her breeding position in 4 out of 13 (30.8%) control nests and in 4 out of 15 (26.7%) joiner-removal nests (x 2 with Yates's correction ¼ 0, d.f. ¼ 1, p ¼ 1). Additionally, established nest residents were no more likely to leave their nests after a joining event in control compared to joiner-removal nests (Mann -Whitney U test: Nest success, measured as date of worker maturation and as brood development at worker maturation, was not affected by treatment or by a switch in the dominant breeder's identity. Only the number of nest residents and brood development at the time of treatment significantly affected brood development at worker maturation (both effects positive; GLM, y ¼ date of worker emergence; main effects: treatment: t ¼ 20.50, p ¼ 0.78, dominance-usurpation: t ¼ 1.51, p ¼ 0.14, group size: t ¼ 20.94, p ¼ 0.35, brood value at joining: t ¼ 21.69, p ¼ 0.10; y ¼ brood value at worker emergence; main effects: treatment: t ¼ 0.45, p ¼ 0.62, dominance-usurpation: t ¼ 0.79, p ¼ 0.94, group size: t ¼ 3.59, p ¼ 0.0012, brood value at joining: t ¼ 4.64, p , 0.001; n nests ¼ 55).

(c) Behavioural interactions during joining events
New joiners did not spend more time foraging than established subordinates, correcting for rank (GLMM; Gaussian error; y ¼ time spent off the nest; main effects: joiner or subordinate: However, new joiners brought back a higher total value of foraging items than established subordinates, correcting for time spent foraging. In other words, forage value per time unit spent foraging was higher for recent joiners than for established subordinates. The amount of aggression that a joiner received also tended to be positively correlated with foraging return values, while relatedness between the joiner and the established nest residents had no effect on foraging returns (GLMM; negative binomial error; y ¼ total foraging return value; main effects: joiner or subordinate: z ¼ 2.73, p ¼ 0.0063, aggression received: z ¼ 1.71, p ¼ 0.088, time spent off the nest: z ¼ 1.75, p ¼ 0.080, average relatedness between joiner and residents: z ¼ 21.47, p ¼ 0.14). Recent joiners that later became the dominant breeders on their nests worked less hard during video recordings than joiners that remained subordinate (n ¼ 19; y ¼ foraging return value per time unit; Wilcoxon's W ¼ 8.5, p ¼ 0.043).

(d) Established subordinates' alternative options
Of 34 released subordinates, we relocated 18 (52.9%) on their second nesting choices: 10 joined other established nests; three initiated a new nest with each other; four joined nests of unknown ages; and only a single subordinate nested solitarily (taking over an abandoned nest). As with the second-choice joiners, the second nesting choices of released low-ranking subordinates were no different than their first choices with regard to inheritance rank obtained and presence of sisters (ranks: Wilcoxon paired, V ¼ 63.5, p ¼ 0.   We quantified the outside options available to cooperatively breeding paper wasps, P. dominula, and found that at the time of joining a nest, individuals had alternative options that offered potentially high fitness payoffs. After we experimentally removed their first nest choices, joiners' second nesting choices included a more than 1: 4 chance of obtaining the dominant breeding position in a social group, which is the highest payoff possible in this species. This means that outside options offered much greater fitness payoffs than solitary nesting: at our field site, the payoff from solitary nesting is close to zero due to extremely high nest failure rates (more than 90% of solitary nests fail [3,40,53]). This result shows clearly that partner choice in P. dominula has the potential to affect payoff distributions in models predicting the amount of help provided by subordinates [7,21] or the amount of reproduction that dominants might have to concede to retain helpers [14,42,54]. Simply comparing payoffs from observed helping decisions with those from a default solitary breeding option, as is traditionally done, is likely to greatly overestimate the relative benefit of staying and helping in the current group. When high-quality outside options exist, dominants may accept a lower subordinate work effort than traditional models would predict. Hence, we demonstrate that multiplayer models, such as those offered by biological market theory, are more appropriate than traditional models for understanding levels of help in cooperative breeders such as P. dominula [7,9,21,22].

(b) First and second nest choices offered similar payoffs
Joiners and established subordinates did not necessarily have to settle for inferior payoff options, compared with their first nesting choice, when forced to make a second choice, contrary to our predictions. This result suggests that individuals had more than one relatively high-quality option available in the market. We found that direct fitness returns associated with the chances of usurping or inheriting the breeding position, as well as indirect fitness returns from helping a related dominant, were no smaller in second choices than in first nesting choices. We predicted that joiners should evaluate their options and choose the one that offered the highest fitness payoff [10,21,53]. However, joiners may have insufficient information to make this choice: it is probably difficult for an individual to evaluate the exact chances of obtaining the breeding position in all nests in the market, prior to actually joining. Furthermore, there is little evidence that females can discriminate relatedness at the individual level in this species [39,44,55] and thereby preferentially join relatives to maximize inclusive fitness. Indeed, both joiners and subordinates sometimes chose to join nests without sisters despite having sisters in nearby nests. These results suggest that joiners chose one of several options available to them, each offering relatively high payoffs, indicating that the biological market is large [22].

(c) Rejecting a joiner may be costly for dominant breeders
Experimentally removed joiners could not easily be replaced with new ones, suggesting that there is not an unlimited pool of potential joiners in the population (as was also found in Polistes carolina [20]). Additionally, nest success increased with the number of nest residents, substantiating previous findings that larger groups fare better [3,40] and that it is in the interest of dominants to accept joiners, particularly related ones, in order to increase group size. Taken together, these results suggest that, as we predicted, it may be costly for dominants to reject joiners. Supporting this result, we found that original dominants were no more likely to lose their dominant breeding positions when joiners were allowed to stay, compared with when joiners were removed. Hence, by accepting a joiner, an original dominant does not necessarily incur a cost in terms of an increased risk of nest usurpation, as she already faces a risk of losing her breeding position to one of her established subordinates. Allowing joiners to stay also did not generally make established subordinates more likely to leave. We thus propose that a dominant cannot afford to be too 'choosy' when presented with potential joiners: it is in her interest to increase group size [3], so long as the risk of the joiner usurping dominance is not too high. This potentially makes joiners the 'choosers' in the market [10], so that dominants are effectively competing with each other to attract a limited supply of joiners. Dominants may therefore be prepared to accept a reduced workload from subordinates in order to retain them when competition for help increases in the population [22].

(d) Joiners may pay for group membership
Rather than using aggression, joiners may have used appeasement in the form of 'pay to stay' in order to become accepted in their new nests. Within the first few days of joining, we found that joiners provided higher-value forage than other subordinates on their nests, perhaps to 'pay' for acceptance by the group. Furthermore, joiners were not involved in a disproportionate number of aggressive interactions, contrary to what would be expected if they were 'forcing' their acceptance as new residents. These results render support for the hypothesis that subordinates trade helping behaviour in return for group membership [15]: when first arriving at a nest, a joiner may need to prove her worth and convey she does not represent a high risk of usurping the breeding position [15,32].
An alternative to the general idea that a joiner works in exchange for group membership is that she works simply because any investment in the nest would directly benefit her if she later took over the dominant breeding position herself ('group augmentation' [56]). However, our findings do not support the hypothesis that subordinates are maximizing only group augmentation benefits, because joiners that later became dominants worked less hard than those that remained subordinate. This is consistent with previous findings that group members higher up the hierarchy, and therefore more likely to inherit the dominant position, in fact work less hard than lower-ranked subordinates [45].

(e) Sampling costs and prospecting
Given that experimentally presented foreign conspecifics are normally attacked by nest residents [44], the lack of aggression towards new joiners suggests that residents may have already been familiar with joiners, perhaps through previous visits by joiners to establish familiarity via 'prospecting rspb.royalsocietypublishing.org Proc. R. Soc. B 284: 20170904 behaviour' [11,57]. Individuals in cooperative species may benefit from maintaining a social network outside their current groups by visiting and familiarizing themselves with members of other groups. This prospecting behaviour can provide them with information about whether betweengroup dispersal would be beneficial, and can maximize their chances of being accepted in the new group, should they be expelled from their current group, choose to leave or if their nest fails [11,57,58]. Indeed, nests fail at high rates in P. dominula [3,40], and prospecting behaviour may be common: we spotted 12.1% of marked wasps on at least two different nests. This number is similar to previous studies of the same population (approx. 16% in [40] and approx. 14% in [22]).
However, prospecting behaviour is likely to be costly [59] as visiting other groups requires time and energy that could otherwise be spent foraging. These costs, called sampling costs or searching costs in biological market terms [9,10], are likely to limit the number of groups a subordinate wasp can maintain in its social network. Second choice joiners mainly chose options that were nearby. This may partly be because nests containing genetic relatives tend to be nearby, but greater costs of prospecting further afield could also contribute. In a scenario where sampling costs are very high, for example in a very low-density population where maintaining peaceful relationships with distant neighbouring groups would pose a high risk of predation or great energetic expenditure, market forces could fail to operate, as there might effectively be no outside options available [10]. Hence, quantifying sampling costs and documenting the actual social networks that individuals gain through prospecting is an important avenue for further studies in this system.

Conclusion (a) Invoking biological market models to include outside options
Our key finding is that in a cooperatively breeding paper wasp, P. dominula, both new joiners and established subordinates have alternative nesting options that offer fitness payoffs comparable with their first nest choices and that are higher than the payoff through solitary nesting. The existence of multiple options with similar payoffs has important implications for the conditions that subordinates should accept in their groups; or in biological market terms, the deal settled on between trading partners. For example, high-quality outside options will affect the trade value of helping behaviour and therefore influence how much help subordinates are prepared to provide with rearing the dominant's offspring [22]. Outside options may also determine whether subordinates should demand a share of the reproduction in return for their services in species where reproductive concessions are likely to occur [23,60,61]. Hence, our findings clearly suggest that biological market models are indeed necessary for understanding helping behaviour in P. dominula. This result is relevant for a wide range of cooperatively breeding species where successful dispersal among groups occurs (for example cichlids [11], carrion crows [62], dwarf mongooses [60], baboons [63]). Unlike traditional models, which assume that a subordinate's only alternative is solitary breeding, market models allow for partner choice, partner switching and competition for partners [9,10]. To conclude, traditional cooperative theory and reproductive skew models are therefore likely to overestimate subordinates' propensity to stay and help in their group, overestimate the level of help that they provide, and perhaps underestimate the level of reproductive concession the dominant should offer her helpers. Future studies should identify and quantify the alternative options available, and include these in models predicting the rate of exchange of cooperative behaviours within groups of cooperatively breeding species.