Indirect reciprocity can overcome free-rider problems on costly moral assessment

Indirect reciprocity is one of the major mechanisms of the evolution of cooperation. Because constant monitoring and accurate evaluation in moral assessments tend to be costly, indirect reciprocity can be exploited by cost evaders. A recent study crucially showed that a cooperative state achieved by indirect reciprocators is easily destabilized by cost evaders in the case with no supportive mechanism. Here, we present a simple and widely applicable solution that considers pre-assessment of cost evaders. In the pre-assessment, those who fail to pay for costly assessment systems are assigned a nasty image that leads to them being rejected by discriminators. We demonstrate that considering the pre-assessment can crucially stabilize reciprocal cooperation for a broad range of indirect reciprocity models. In particular for the most leading social norms, we analyse the conditions under which a prosocial state becomes locally stable.


Introduction
Natural selection disfavours indirect reciprocity unless the costs of observation and assessment are negligible [1].According to proper social norms that distinguish good from evil, such as image-scoring norm [2], indirect reciprocity can promote cooperation even in large populations [3].However, making moral assessments takes time and effort.
Discriminators who incur no assessment cost thus appear as free riders that erode a cooperative state achieved by discriminators who incur the costs.Despite the advance of indirect reciprocity, the crucial question remains unsolved [4]: How can cooperation through indirect reciprocity be maintained when considering the costs associated with the assessment system?
To address this crucial question, we focus on the fact that this puzzling situation is closely related to the second-order free-rider problem in costly punishment [1].The evolution of costly punishment, in striking contrast to indirect reciprocity, has been given much more attention over the last decades.In tackling the second-order free-rider problem, previous study significantly examined pool punishment [5][6][7][8][9][10].The key aspect of pool punishment is its proactive mechanism to detect second-order free riders through unconditional prepayment.The mechanism paves the way for effectively punishing second-order free riders.
In this paper we apply the essence of the pool-punishment mechanism to fix the issue of the costly moral assessment.In the next section, we introduce a basic model of indirect reciprocity and the known negative outcome from considering the assessment costs.In the Results section we show how adopting a proactive assessment mechanism can improve the outcome.

Materials and methods
We build upon the standard framework for the evolution of indirect reciprocity by reputation [11,12].Using the framework, a strategy for discriminators is given by an assessment rule combined with an action rule.We base indirect reciprocity on the giving game, which is a two-player game in which one player acts as a donor and the other a recipient.The donor can choose to help the recipient by giving benefits b > 0 at personal cost c > 0 or not to help.We consider the following implementation error: a player who has intended to help involuntarily fails to do so with a probability e [1,13].
We start with a basic model in which each individual is endowed with a binary image score of 'good' or 'bad'.It is assumed that the discriminator's action rule is to help a good recipient or not to help a bad recipient.After observing every giving game, a unique assessment system assigns the donor's image by following a specific assessment rule.We assume that all discriminators share the same list of individual image scores provided by the assessment system.We later consider in particular the second-order assessment rule, which is a function of the donor's last action and the recipient's last image (Table 1).
To study the evolution of discriminators, we respect a continuous-entry model: an individual's birth and death sometimes happen, and this changes the strategy distribution in the population [14].We assume that in one's lifetime an individual infinitely plays the oneround giving game with different opponents.We consider infinitely large populations and analyse the replicator dynamics [15] for the following four strategies: 1) Paying discriminator [Z] is willing to help a good recipient and refuses to help a bad recipient in the giving game.Also s/he is willing to pay for the assessment cost k > 0 .2) Evading discriminator [W] similarly acts as a paying discriminator in the giving game, except that s/he is not willing to pay for the assessment cost.3) Cooperator [X] unconditionally intends to help a potential recipient, and 4) Defector [Y] unconditionally intends not to help a potential recipient.Both cooperator and defector are not willing to pay for the assessment cost.We denote by x , y , z and w the frequencies of cooperators, defectors and paying and evading discriminators, respectively.The replicator dynamics for these strategies are described as dn dt = n(P S − P) , where n is the frequency of strategy S (= X, Y, Z, W), P S is the expected payoff given by the limit in the mean of the payoff per round for strategy S and P is the average payoff over the population, given by xP X + yP Y + zP Z + wP W .
To formalize the expected payoffs, we denote by g S the probability that a recipient with strategy S is helped by a given discriminator.In the basic model this is identical to the fraction of good players within all S strategists.Let g be the population average of g S , thus The population size is very large, so we may assume that the population configuration for g S does not change between the consecutive one-round giving games [16].Thus, the expected payoffs are described as (1) We note that in the basic model either paying or evading discriminators intend to help a potential recipient who has a good image, thus leading to g Z = g W .This results in paying discriminators being worse off than evading discriminators.Substituting this into Eq.( 1) yields For any degree of the assessment cost k , evading discriminators dominate paying discriminators in the interior state space, and thus the population in the end attains a state that excludes paying discriminators.In the absence of cost payers, the assessment system cannot be established.Consequently, cooperation in that case would vanish without discrimination.

Results
The basic model reveals that considering cost evaders destabilizes indirect reciprocity irrespective of the assessment rule, as shown in previous work [1].To stabilize indirect reciprocity, we examine an institutional variant of the basic model.As a first step, we extend the basic model to a two-stage game in which one round consists of the stage of payment for the observation costs followed by the stage of the giving game, which is the same as in the basic model.The first stage offers an opportunity to transfer some fees to a central account as in automatic utility payments.
The essential idea is to specifically assess the second-order free rider.We consider a different binary moral code 'nice' or 'nasty'.The (unique) assessment system assigns a nice image to an individual if s/he pays the costs in the first stage, otherwise that individual is assigned a nasty image.In evaluating the donor's action of the giving game, as the first step we simply apply the existing assessment framework to the second stage, as in the basic model.
We keep the four strategies, cooperators, defectors, paying discriminators and evading discriminators, as before and assume that in the first stage, paying discriminators are willing to pay but the remaining cooperators, defectors and evading discriminators are not.We also modify the discriminator's action rule for the giving game as follows: either paying or evading discriminators give help if a potential recipient has a good and nice image, or otherwise (if bad or nasty), refuse help.
The extra assessment by the utilities payment system seriously lowers the image score for second-order free riders.For analytical simplicity, we assume that the utilities payment system is so perfect that no assessment error occurs for the first stage.All of evading strategies: cooperators, defectors and evading discriminators (X, Y and W), therefore, are necessarily assessed as nasty.This yields g X = g Y = g W = 0 .(Note that in the variant, g S equals the probability of good and nice players.)Thus, Eq. ( 1) becomes ( It is clear that P Y ≥ P W ≥ P X .To understand when the homogeneous state of paying discriminators z = 1 becomes locally stable, it is enough to check if P Z − P Y > 0 in the vicinity of z = 1 on the face x = w = 0 .This yields With suitable assessment rules, it is possible to have that g Z > 0 in the vicinity of z = 1.In this case, the node z = 1 turns into a locally stable equilibrium when the net benefit b− c is sufficiently large compared to the assessment cost k . Finally, we demonstrate how the variant improves the results for some of the most leading assessment rules.We examine simple standing [13,16] and stern judging [17], the only two second-order assessment rules in the leading eight norms [10,11].According to the discriminator's action rule in the variant, we extend simple standing and stern judging as in table 1.These rules assign a good image to those who help a good and nice recipient with no implementation error (probability (1− e)g ) and also a good image to those who refuse to help a bad or nasty recipient (probability 1− g ).By assumption of the image dynamics, the sum of these probabilities should equal g Z .Considering also g = g Z z then leads to the recursive equation for g Z , g Z = (1− e)g Z z + (1− g Z z) .This yields g Z = 1/ (1+ ez) .Hence, the necessary and sufficient condition for the homogeneous state of paying discriminators ( z = 1) to be locally stable either under the simple-standing or stern-judging rule is Figure 1a shows the basin of attraction for z = 1, and figure 1b depicts the flow on the boundary faces of the state space under simple standing.If we assume assessment errors in the first stage, the image dynamics become more complicated but the main results remain qualitatively unchanged-paying discriminators can stabilise with the pre-assessment of cost evaders (electronic supplementary material, S1).

Discussion
Since the definitive 2013 work by Suzuki and Kimura [1], the evolution of indirect reciprocity relying on costly assessment systems has been explicitly recognized as one of the inevitable issues that challenge the advance of indirect reciprocity [4].To address the issue, we considered a simple pre-assessment mechanism that is set prior to the primary game in order to detect and label cost evaders.We then demonstrated that the mechanism considered leads to stabilizing costly indirect reciprocity under the most leading social norms, simple standing and stern judging.
Our results are potentially applicable to a broad range of existing indirect reciprocity models, such as tolerant scoring [18], group scoring [19], reputation-based punishment [20], mixed public and private interactions [21], optional interactions [22] and finite populations [23].On the one hand, managing more complicated assessment systems, such as in [18][19][20][21], would be more costly, and thus it is worth considering pre-assessment mechanisms for reducing the temptation to evade cost sharing.On the other hand, as in the case of pool punishment [8], jointly considering optional interactions [22] and finite populations [23] might facilitate establishing pre-assessment mechanisms.
Another promising avenue for future studies would be to explore costly indirect reciprocity on more realistic structured populations.Recent studies using structured populations suggest the importance of cooperator assortment based on reputation [24,25].
However, little is known about how information cost affects reputation-based reciprocity on a network.In the case of the second-order free-rider problem in costly punishment, considering the locality of interactions among players can solve the problem by separating costly punishers from second-order free riders [26].Similarly, the extension to structured populations may lead to significantly different outcomes for paying and evading discriminators.
We left out an advanced issue of analysing nonlinking discriminators [27] who act as paying discriminators yet are willing to help cost evaders with a good image.Nonlinking discriminators can invade paying discriminators by neutral drift.The preliminary results indicate that considering implementation or assessment errors for the first stage can lead paying discriminators to become better off than nonlinking ones, as in fixing neutral drift between conditional and unconditional cooperators [13].Further investigation is planned in future work.
We note that prepayment for assessment systems can be viewed as a kind of contribution to collective action.Thus, our results corroborate those of previous studies on two-stage games in which reciprocal behaviours in the second stage are linked to a collective action in the first stage.For instance, Panchanathan and Boyd showed that collective action in the first stage can be maintained by considering a shunning strategy that in the second stage withholds help for those who failed to contribute in the first stage [28].Together, the present results further imply that such a proactive social mechanism that can discriminate those who deserve to enter social exchange and reciprocal norms within social exchange may evolve jointly.We examine assessment errors for the first stage of the variant model considered in the main text.We explore the conditions for the homogeneous state of paying discriminators [Z] with z = 1 to become stable under simple standing and stern judging.We check when paying discriminators become better off than the other three strategies: cooperators, defectors and evading discriminators.
First, we analyse the frequency of good and nice players among S-strategists (S = X, Y, Z or W), g S .As in the main text, we assume that the degree of g S is unchanged between the consecutive one-round (two-stage) games.We note that by definition the only difference between the rules is with respect to how a potential donor is to be assessed when a potential recipient is bad or nasty and the donor's action is not to help, in which case simple standing assigns a good image and stern judging a bad image.
We denote by e 1 the probability of a first-stage assessment error in which the assessment system involuntarily assesses a paying player (who should have been nice) as nasty or an evading player (who should have been nasty) as nice.
For simple standing, g S is given by is sufficient for z = 1 to also be stable for a sufficiently small degree of e 1 in either case of simple standing or stern judging.

Figure caption Figure 1 .
Figure caption

Table caption Table 1 .
How second-order rules make moral assessments in giving games with pre-