The emotion system promotes diversity and evolvability

Studies on the relationship between the optimal phenotype and its environment have had limited focus on genotype-to-phenotype pathways and their evolutionary consequences. Here, we study how multi-layered trait architecture and its associated constraints prescribe diversity. Using an idealized model of the emotion system in fish, we find that trait architecture yields genetic and phenotypic diversity even in absence of frequency-dependent selection or environmental variation. That is, for a given environment, phenotype frequency distributions are predictable while gene pools are not. The conservation of phenotypic traits among these genetically different populations is due to the multi-layered trait architecture, in which one adaptation at a higher architectural level can be achieved by several different adaptations at a lower level. Our results emphasize the role of convergent evolution and the organismal level of selection. While trait architecture makes individuals more constrained than what has been assumed in optimization theory, the resulting populations are genetically more diverse and adaptable. The emotion system in animals may thus have evolved by natural selection because it simultaneously enhances three important functions, the behavioural robustness of individuals, the evolvability of gene pools and the rate of evolutionary innovation at several architectural levels.


Introduction
In evolutionary ecology, there has been a focus on finding the optimal phenotype for a given environment, with Optimal Foraging Theory [1,2], Life History Theory [3,4], Game Theory [5,6] and State-Dependent Behavioural and Life History Theory [7,8] as the major methodologies. However, with few exceptions (e.g. [8,9]), these generally do not consider the fitness of sub-optimal phenotypes, the impacts of adaptive phenotypes on the genome and gene pool, or the constraints evolutionary pathways make on the adaptive solution [10].
There are at least two evolutionary obstacles to arriving at the optimal phenotype: the evolution of the gene pool and the formation of the phenotype from the genotype. First, the fitness landscape may not have a single peak that is easily accessible from all starting points. A rugged or holey landscape with many solutions creates a path-dependence [9,11]-a historical contingencyin the process of adaptation. In addition, if frequency-dependence is important in selection, then the fitness landscape changes according to the present state of the gene pool, such that the location and surroundings of peaks and paths between them may be constantly changing. These factors suggest that the process of adaptation, even in the same physical environment, may end up at different peaks in the fitness landscape. Second, according to the 'phenotypic gambit' [12], the assumption that natural selection leads to the optimal phenotype further assumes a direct correspondence between an unconstrained genotype and the behaviour or trait that is directly relevant for fitness [13]. Optimization approaches thus usually overlook all the mechanisms between the strength of selection and the behaviour or other phenotypic trait assumed to be optimal, for example genetic coding, sensing, cognition and decision-making [14][15][16].
While behavioural ecologists have used the phenotypic gambit as an argument for what to expect at the phenotypic level [12,17], one can also ask about the consequences for the gene pool of adding mechanistic layers between the genotype and the phenotype [18][19][20] and the consequence in terms of predictability and diversity for these intermediary layers themselves to be under selection [21][22][23]. We investigate this question in an idealized model of the emotion system [24] that links genes to phenotype and study evolutionary adaptation in environments that include possibilities for age-, state-, density-and frequency-dependent adaptation and behaviour. This allows us to simultaneously investigate whether trait architecture or environmental factors alter the tendency of gene pools to arrive at the same peaks in the fitness landscape, the resulting phenotypic diversity in the adapted populations and the characteristics of gene pools of adapted populations with trait architecture.
We find that a dominant trait architecture in higher animals, the emotion system, is itself sufficient for generating diversity, and that neither environmental variation nor complexity will prevent the population from arriving at predictable phenotypes. We also show that the predictability decreases as one moves from life-history traits to gene pools. Our results indicate that trait architecture gives individuals the flexibility to respond appropriately to new situations in the short term (a lifetime), but also that architecture has longterm evolutionary consequences as it leads to diverse gene pools with high adaptability to a wide range of new challenges.

Material and methods
The emotion system is central in converting sensory information via decision-making into behaviour, at least in vertebrates [25 -28]. It is a system in a behavioural sense, but it is not a neurobiological unit. While the Euler -Lotka equation [29] and later the method of dynamic optimization [8] have been proposed as models for the 'common currency'-mechanism for evolutionary adaptations by natural selection, the emotion system acts as a proximate common currency mechanism: it is evolutionarily adapted to integrate and evaluate information of widely different usage for the organism.
We use a model of a fish population where the trait architecture includes genetics, physiology, emotion, behaviour and reproduction with inheritance [24]. In this model, perception, neuronal responses and developmental modulation are influenced by the genome and determine the individual's 'global organismic state' [28] which restricts its attention [30,31] and constrains its behavioural choices [28] (figure 1).
Our model describes fish in a pelagic environment, so that the behavioural alternatives are to move one level up or down or to remain at the same depth. Generations are non-overlapping and last for seven diel cycles of continuous surface light change.
(Hence, we have condensed a year to 7 days.) There are 200 time steps per day where the fish determines its depth according to the processes illustrated in figure 1. Light decays with depth, impacting both predation risk and feeding opportunity, and thus creates a temporally and spatially variable environment [32,33]. Crowding reduces individual mortality risk but also generally increases competition for food [34][35][36]. See detailed descriptions in the electronic supplementary material of [24]. (Although unseen by most people, mesopelagic fishes are the most abundant fishes in the ocean [37].) The emotion model considers two attention-competing survival circuits [28]. The dominating circuit sets the organism in a global organismic state, which in our case is fear or hunger. Individuals whose emotion system (figure 1) has made them afraid, regard light as a risk enhancer and conspecifics as risk dilutors, while hungry fish are attracted to food and regard conspecifics as competitors [24].
The form in which genes impact the emotion system is central to the model. Thus, while all hungry individuals regard conspecifics as negative in the evaluation of the quality of a potential resource (figure 1), some may pay little attention to conspecifics while others may strongly avoid them [24]. We assume that there are two genes in each of the nine neuronal response functions that link a sensory perception of the environment or of the state of the organism to emotions and behaviours in the model ( figure 1). Further, the organism has one sex-determination gene and four genes that modulate the impact of development (through body mass) on fear and hunger. Hence, each haploid fish has a unique set of alleles of 23 genes inherited from its parents. The two genes in a neuronal response together form a chromosome, which is subject to mutation but not recombination. The sexdetermination gene and the four developmental genes are located on one chromosome. As a consequence, developmental modulation may become sex-differentiated, while a neuronal response chromosome will alter between being in female or male individuals. Selection differs among the sexes as a female who survives until the last time step of a generation produces eggs in proportion to her body mass and then chooses the largest of three randomly encountered surviving males as father for her eggs. Hence, there is larger variation in parenthood in males, driving selection for larger body mass via the genes for developmental modulation, thus less fear and higher mortality. Examples are the spikes in mortality in males, explained in fig. 4 in [24]. The model is explained in detail according to the ODD (overview, design concepts and details) protocol [38,39] in [24].
We studied the effects of environmental and organismal complexity in four scenarios (table 1), ranging from no environmental variation in time and space ('game only') and no density-or frequency-dependent processes ('no game'), via an environment that is exactly the same in each generation ('repetitive environment'), to rich environmental variation and density-dependency ('full' scenario). For each scenario, we ran 30 simulations over 20 000 generations, initiated with 10 000 individuals with random alleles. The mutation risk of one gene in one individual is 0.001. Ninety per cent of mutations are to one of the nearest alleles, and 10% are to a random allele. This sums to 200 000 mutations per gene over a simulation. Behavioural and lifehistory phenotypes did not converge in 13 of the 30 simulations in the 'game only' scenario, even after 20 000 generations. We therefore also ran 10 simulations for more than 100 000 generations in the most extreme scenarios ('game only' and 'full').
In the 'full' scenario, we only used generations with 'standard environment' (table 2) to compare simulations. We collected the statistics of genotypes (details in the electronic supplementary material, figure S1) from all individuals in the 200 last 'standard environment' generations in each simulation. The 'no game' and 'game only' scenarios do not include this standard environment but have other environments that repeat each generation (table 1). For these scenarios, we used results from the 200 last generations. We measured non-genetic data at six fixed periods or points through life for each individual in the final generation, except for life-history data which were measured only at the end of the generation. Thus, for each scenario, our results are based on 150 000 observations of each sex-specific life-history parameter, 900 000 observations of other phenotypic parameters, 30 million observations of the sex-specific developmental genes and 60 million observations of all the other genetic parameters. We quantified within-population diversity for 88 individual variables for the 30 populations in each scenario by the average within-generation coefficient of variation (CV) over these 200 generations.
We estimated evolutionary diversity among the populations within a scenario by interquartile range and total range of within-population averages for the same variables, after normalizing each population average against the average among populations. This standardized the variation relative to an average of 1. We excluded the 13 populations that did not arrive at the scenario-typical phenotypic solutions (i.e. the adaptive peak) in the 'game only' scenario from the analysis of variation within and among adapted populations.

Results
Phenotypic diversity evolved within and among all populations, and this diversity was lowest for life-history traits and increased as one moves via physiological and behavioural traits and global organismic state through to the genotype. In the following, we first describe the 'full scenario' as basis for comparison, and then relate it to the other scenarios to explain patterns of diversity in light of trait architecture.
(a) Phenotypic convergence in scenarios with high fidelity to nature In the scenario with the highest fidelity to the situation of small planktivorous fish-the 'full' scenario with densitydependency, frequency-dependency and rich environmental variation-the evolving populations went through an initial phase of rapid adaptive changes, best seen as an increase in the egg production (figure 2a). This initial phase was usually shorter than 2000 generations but varied in length between simulations (electronic supplementary material, figure S2). After the initial phase, the allele frequencies still changed continuously, but this had little impact on total fecundity of the population or average phenotypic characters (figure 2). After this initial phase, all phenotypic characters (final body mass, mortality, crowding and average depth) converged across the simulations ( figure 2 and , respectively. (The subscript A indicates that these are neuronal responses involved in the emotional appraisal.) The emotional appraisal ends with the stronger of these determining the global organismic state of the organism [28]. The emotional response is specific to the global organismic state and includes physiology and behaviour. The physiological response to this emotional appraisal is an attention restriction of the organism. In the processing of its behaviours, it thus re-evaluates a subset of its sensory information in its current depth z and the immediate depths above and below. Hungry individuals (neuronal response subscript H ) will value food as positive and competitors as negative and ignore other information: max zÀ1,z,zþ1 (R H food À R H conspecifics ). Frightened individuals (subscript F ) will regard light as negative and conspecifics as positive: max zÀ1,z,zþ1 (R F conspecifics À R F light ). When the relevant behaviour is executed, the animal starts over on top with new emotional stimuli. Adapted from [24].
rspb.royalsocietypublishing.org Proc. R. Soc. B 281: 20141096 contains diel and vertical environmental variation as well as density-dependent effects of the presence of conspecifics (table 2). The scenarios without density-dependence and frequencydependence ('no game') and environmental variation ('game only', see below) had the lowest within-population variation in all parameters (figure 3). However, there was genetic, global organismic state and phenotypic variation both within and between all populations.
(b) Slow or no convergence in the pure game scenario In 10 long-term simulations of the 'game only' scenario, the common phenotypic distribution was reached before 20 000 generations in seven simulations, and after 26 000 and 106 000 generations in two others, while one simulation had not converged to this solution when it terminated after 105 000 generations. Thus, variation after 20 000 generations in the 30 simulations in the 'game only' scenario (electronic supplementary material, figures S2-S5) was caused partly by a delay in arriving at the common phenotype distribution, and all these deviant simulations were still at a phenotype space with lower fitness (population egg production; electronic supplementary material, figure S2) at the end of the simulation.

(c) Within-population variation
In all scenarios, the adapted populations contained individual diversity in life-history, physiology, behaviour, global   Neither frequency-dependence nor environmental variation were needed to maintain this diversity, although variation within populations was lowest in the scenarios where spatial and temporal variation was removed ('game only' scenario) or where the behaviour of others did not affect food availability or predation risk, thus making the same vertical trajectory optimal for all individuals ('no game' scenario). Even in the 'no game' environment, in which there is one single, optimal behavioural solution that fits all individuals, there were differences in final body mass, tendency of being afraid and depth at reproduction (figure 4; electronic supplementary material, figure S2-S5). These differences were caused by coexistence of several alternative neuronal responses to the same environmental stimuli and lead to individual variation in behaviour and, consequently, in phenotypes.

(d) Among-population variation
The patterns in behaviours and life histories converged between simulations within each scenario, whereas global organismic state and genetic solutions did not, as shown for the full scenario in figures 2 and 5 and for all scenarios in figure 3 and the electronic supplementary material, figures S2-S5.
While the adaptive solutions at higher levels in the architectural complex (behaviours, physiologies and life histories) were predictable, meaning that each simulation of a scenario produced similar frequency-distributions of parameters for that environment, the solutions at lower levels (genetics and global organismic state) could not be predicted from environmental factors. This is clear from the low CV among populations in figure 3 Table 2. Environmental variation within one generation in some of the scenarios.  rspb.royalsocietypublishing.org Proc. R. Soc. B 281: 20141096 environmental variation nor frequency-dependence is a prerequisite for diversity. The flexibility granted by architecture is evident in the increasing variation from life-history traits to genes when one compares across populations. This shows that the critique emphasized by the phenotypic gambit [12] and the behavioural gambit [16] is very relevant, because simplified models without intermediate layers will underestimate natural variability at lower levels of biological organisation, including the genotypic.
The partly random historic path of the evolving gene pool itself becomes important in determining its future [9,40,41], as for other systems when external forcing becomes weak. This is seen in the accumulated differences in what makes the red and blue populations in figure 5 afraid. The strong frequency-and density-dependent selection in the environment in most scenarios leads to the array of co-adapted phenotypes rather than a single best strategy. However, even in the scenarios without these external forces, the populations arrived at a diversity of both phenotypes and genotypes.

(b) Drivers for diversity
As is true in natural populations, the diversity seen in our study has several causes. Diversity between phenotypes is created in each generation by sexual reproduction, where each individual genome is a mix of its parents' genomes and thus may be viewed as a new evolutionary experiment never seen before. Diversity is further increased by numerous stochastic events at the level of the individual [24], which is also important for variation in state-dependent life-history models [7,8]. In addition, a temporally varying fitness landscape, where ephemeral hills and valleys constantly emerge, enables coexistence of multiple strategies and solutions. This is partly caused by frequency-dependent games between strategies [24], through which competition and mutualism affect behaviour, growth and mortality. In addition, a variable environment amplifies gene -environment interactions through shifting periods where some allele combinations prosper and others decline [24].
Variation increases and predictability decreases from phenotype to genotype because there is a complex and nonlinear mapping between genotype and phenotype (figure 1). For example, the likelihood of becoming afraid can change through any of the three neuronal responses related to fear or the two related to hunger, or the four developmental modulator genes. Similar nonlinearity applies to other brain configurations, e.g. neural networks [42][43][44][45]. This nonlinearity gives rich opportunity for evolvability [18][19][20] and evolutionary innovations [21][22][23] at the genetic, neurobiological and emotional levels. The architecture defines a vast space of possible individual and population configurations (e.g. figure 5), as is observed in evo-devo [46,47] and phenotypic plasticity [48].
Between populations, the existence or extinction of specific strategies lead to path-dependency [9], whereby a particular population ends up with only one of many possible population configurations at a particular time. The multiple sources for maintenance of biological variation lead to rich variation within and among populations in a wide variety of conditions, and by the different scenario experiments we show that this pattern is quite robust.
Together, these mechanisms yield a unique historic contingency with multiple opportunities for diverging selection among populations. This highlights two sources of biodiversity.
Path-dependency gives among-population variation in averages and a potential for higher order diversity, like speciation, whereas within-population variation originates from architecture and may be strengthened by spatio-temporal games among coexisting strategies. As selective freedom is highest for genes, diversity among populations decreases from genes, via emotions, to behaviour, physiology and life-history traits (figure 3 and the electronic supplementary material, figure S1). Experimental studies have found large behavioural differences within and among natural fish populations [49][50][51] and our work suggests that the genetic, neurobiological and emotional basis for such differences can accumulate even in the absence of long-term environmental differences.

(c) Higher order effects of architecture
The variation among populations in genetics, neurobiology and emotions, contrasted with the consistency among populations in growth, space use and life history, indicates that a diversity of life forms can adapt to the same environment by evolving different genetic, neurobiological and emotional innovations. It also emphasizes the role of the organismal level of selection [52], as adaptation to the environment does not depend on particular gene pools.
Gould & Lewontin [53] objected to the adaptive phenotype as a 'Bauplan'. Their argument was that one cannot expect evolution to optimize all sorts of phenotypic characters, as observed by Giske et al. [24]. But the Bauplan also comes with evolutionary advantages. One is that architecturedriven diversity among populations in a region may be a mechanism for preadaptation [54,55] and exaptation [56], because this diversity increases the possibility that some of these populations will contain genetic variation that may become important for future adaptability towards new challenges [57]. Another advantage is that architecture by itself enhances genetic variability in the population and thereby reduces the risk that the gene pool will get stuck at a local sub-optimal peak in the fitness landscape.
Architecture will cause individual variation and at the same time population-level phenotypic similarity. Trait architecture makes organisms more constrained than what is usually assumed in optimization theory, in the sense that genetics limit phenotypic plasticity [24]. However, trait architecture also makes populations more diverse and evolvable [18][19][20]. Thus, it is likely that the emotion system has promoted itself through evolution not only by enhancing the survival of the individual but also through the evolvability of the gene pool. The trait architecture of emotion in animals [28] gives flexible individuals that are able to respond adequately to a wide range of familiar and unfamiliar situations [58,59], and also robust gene pools adaptable to a wide range of new challenges [20]. Trait architecture, and its consequence, pathdependency, may be important factors creating and sustaining diversity, as well as in determining evolutionary winners.