Identifying innovation in laboratory studies of cultural evolution: rates of retention and measures of adaptation

(a) Experimental studies of cultural evolution

Although experimental studies of cultural evolution may take a variety of forms, we believe that all designs share certain unifying features that are worth outlining here. First, in contrast to more typical psychological experiments that concern how a single individual performs on a task, or sometimes how one individual learns from another, in studies of cultural evolution a single replicate within an experiment consists of multiple (three or more) participants. In this way, these designs capture the repeated occurrences of social learning involved in cultural change, as opposed to one-off cases of individual learning or social learning in general. Second, within each replicate, participants have some form of access to information about the solutions or responses of other members of the same replicate. The exact nature of the information available may vary, but can include direct observation, verbal report or stored information about solutions or responses presented remotely, i.e. in the absence of their progenitor. Finally, all studies involve a measure that is repeated successively, the overall aim being to describe the nature and/or direction of change that arises within sequences of measurements.

As an example, a simple cultural evolution study might involve one participant completing a task, set by the experimenter, in front of an observer. Upon completion of the task, the first participant's performance is evaluated, and the observer takes over the role of task completion, with a new participant arriving to take the role of observer. This would generally continue for a pre-specified number of iterations, which together would represent a single replicate within the overall experimental design. In an example such as this, any changes in the task scores would likely represent a key measure of interest.

It is not within the scope of this review to provide an exhaustive catalogue of such studies (and indeed more comprehensive reviews of the literature can be found elsewhere [2,3]). We instead intend to provide an overview of dominant approaches, using illustrative examples of particular studies where relevant, with particular focus on those that permit insights into the role of innovation.

In terms of the methods of structuring the multi-participant replicates in cultural evolution experiments, some common approaches have been described in the previous literature. Mesoudi [4] distinguished three main approaches, labelling these as the transmission chain method, the constant group method and the replacement method. In transmission chain studies, participants take part in the experimental task one at a time, in strict succession, receiving information only from their immediate predecessor. In contrast, in studies using the constant group method all members of a replicate take part simultaneously, so group membership is fixed, and there is no addition of naive participants. Although in all constant group studies it is possible to learn from any other member of the group, a further distinction can be drawn between one type of design, in which the exchange of information is unrestricted (sometimes referred to in the literature as ‘open diffusion’ [5]), and those where information exchange is under control within the experiment. Finally, the replacement method incorporates elements of both transmission and constant group methods: in these studies, a small group of participants complete the experimental task simultaneously (as with constant groups), but experienced members of the group are replaced at regular intervals by naive newcomers, by way of simulating generational succession within a population. For this reason, such approaches are also sometimes referred to as microsocieties [6,7]. Using this method, it is therefore possible to ensure complete turnover of group membership while retaining some flexibility over whom participants can learn from.

Within the current review, we intend to restrict our discussion to those studies that incorporate generational turnover as part of the design (i.e. including transmission chains and replacement microsocieties, but excluding studies using the constant group method). Attributing changes that occur within constant groups to the process of cultural evolution (characterized as a Darwinian process consisting of the selective retention of favourable socially learnt cultural variants as well as a variety of non-selective processes such as drift, migration and invention [8–10]) is relatively problematic, because individual learning processes (particularly feedback from trial and error) will typically tend to result in directional changes in behaviour over time. This makes it difficult to determine the extent to which any such changes have occurred as a consequence of cultural evolution or merely as an effect of iterative individual learning. Such designs can nonetheless be extremely valuable for certain research questions within this field (e.g. for comparing the effects of different group sizes, to understand the additive effects of social information on individual learning [11] or for exploring how cultural traditions, once formed, are actually spread through populations [5]). However, for the purposes of identifying innovations (see §2), we feel that transmission chain and replacement designs allow potential heuristics for doing this, which are less readily interpretable in the context of constant group approaches.

(a) Measures of similarity

The usefulness of our definition of innovation rests on the ability to define the degree of similarity between different cultural variants. Similarity has been explicitly quantified in a number of studies, using a range of different methods. There are several reasons why researchers have employed such measures as a dependent variable in their designs. In some cases, the motivation has been to determine whether material becomes more learnable with transmission, as evidenced by decreasing error rates (and increasing similarity) over generations [12–14]. In other cases, similarity estimates have been used to determine whether performance improvements over generations are associated with a pattern of descent with modification, indicative of cultural evolutionary processes [15]. Such measures can also be used to establish whether separate lineages of variants are distinguishable from one another, in a manner characteristic of distinctive cultural traditions [7,15–17].

The precise method used to evaluate similarity between variants is determined largely by the nature of the behaviours in question. For example, in studies using artificial language learning tasks, where the cultural variants being studied are sequences of linguistic symbols (i.e. words or sequences of words), Levenshtein edit distance has been used [12,13]. This metric calculates how similar one string of characters is to another by counting the minimum number of characters that must be substituted, inserted or deleted to transform one string into another, normalized by the length of the longer string. Other studies have used subjective judgements of similarity as assigned by naive raters, by simply asking them to compare two items and indicate how closely they resemble one another; this has the advantage of validity as a direct measure of human perception of resemblance, but has the drawback of being opaque in relation to the source of similarity in terms of which features are shared [7,15,16]. Verhoef et al. [14] used a similarity metric based on the physical acoustic properties of an auditory signal, but derived the weightings assigned to these properties from perceptual ratings obtained in a separate pilot study, thus using an objective measure with accompanying assurance of subjective validity. In other studies, similarity between variants, although not explicitly part of the research design, can sometimes be inferred from other measures used to track retention of particular features of interest (often those that were present in stimulus material presented to the first generation of participants), by considering the number, or proportion, of shared features (e.g. in ‘serial reproduction’ studies [18–21]).

By using such measures of similarity, it seems plausible to quantify the degree of deviation from perfect copying, and therefore the rates of innovation in different experimental studies.

(b) Sources of innovation

Our definition of innovation does not distinguish between different sources of innovations. In the modelling literature, innovations are generally regarded to be a potential outcome of individual learning [22] or of erroneous cultural transmission [8,23], the latter being commonly referred to as mutation. While the exact characterization of an innovation varies between approaches (e.g. sometimes defined as novel to the individual, and in other cases defined more narrowly as novel to the population), their function is very similar: innovations induce the possibility of cultural change into the considered system.

Within studies of cultural evolution, therefore, innovations may similarly arise as a consequence of transmission error, or individual learning (involving intentional invention or modification on the part of the participant). However, in current studies of cultural evolution, it will generally be difficult to distinguish between transmission error and individual learning based on the available data (e.g. the sequence of cultural variants produced in a transmission chain). Potential inferences about the source of innovations will depend on the chosen experimental design (discussed in §3).

This leads us to define innovativeness as a continuum, rather than as a dichotomy, with faithful social transmission and innovation considered as opposite ends of a spectrum of possibilities representing a balance between the two. So for our purposes someone who intended to copy, but who failed and produced something very different from anything to which they had been exposed, would be defined as having innovated. In contrast, an individual who independently conceived of a solution that was highly similar to another solution potentially available to them via social learning would be defined as not having innovated. These simplifying assumptions allow us to operationalize innovation in a way that makes it possible for us to identify it from experimental studies of cultural evolution.

(a) Task aims and incentives

In this section, we discuss two broad categories of cultural studies that differ in terms of the goal as presented to the participants. Specifically, we distinguish between studies requiring accurate reproduction (denoted reproduction goal studies) and studies involving evaluation of performance on a specified task (denoted performance goal studies).

(b) Measures of adaptation

To the best of our knowledge, in all experimental studies of cultural evolution, there is generally some sort of expectation about the nature of the change that repeated transmission is liable to generate. The different measures used, however, will be affected differently by innovations, and in some cases, innovations arising from intentional modification are likely to affect measures differently from those that arise from social learning errors. In the following section, we discuss three broad methods that have been used to measure adaptation in studies of cultural evolution.

(iii) Cultural attractor measures

In a third category of studies, the measure of interest represents a specific property of the transmitted behaviour, which is predicted to increase with transmission as a consequence of this property rendering the material more learnable. The property is therefore assumed to represent some sort of cultural attractor [30] whose presence, or probability, will tend to increase relative to source material provided to the first participant of a chain (in which the attractor would be normally be represented at statistical chance level or below) or in which the degree of representation might be systematically varied [31]. Examples of studies using this kind of measure include artificial language learning studies that predict increases in structural compositionality [12], predictability of grammatical markers [32] or regularization [31]. However, we would also include in this category studies that seek evidence of the emergence of cognitive ‘priors’ over repeated transmission [33,34]. In these studies, participants attempted to infer a function [33] or category membership hypothesis [34] from a set of exemplar data, with their selected function or hypothesis being used to generate exemplar data for the next participant. Over repeated transmission, the functions and hypotheses that increased in probability were those that represented known human learning biases.

These studies have typically emphasized a goal of accurate reproduction for participants assessed by probing their knowledge of the learned material. However, it is also possible to measure these sorts of changes in studies framed in terms of task success (see e.g. [13], for an example of a language evolution study using effective dyadic communication as the participants' goal).

Considering these studies from the perspective of the effects of innovation illustrates an important difference between this type of measure of interest, and those involving simply loss and/or distortion of source material. In studies looking for the emergence of cultural attractors, it is perfectly possible for errors in transmission to result in changes that move in the opposite direction to the prediction. As noted previously, in studies documenting degradation of source material, any kind of loss or distortion effectively generates change in the predicted direction. Nonetheless, in studies measuring the presence of presumed cultural attractors, it is still quite likely that increased error rates will tend to increase the cultural change in the direction of the proposed attractor, because it is assumed to be the result of some kind of cognitive bias.

Figure 1 provides an overview of the different categories of cultural evolution experiments we have outlined here, i.e. in relation to the participant's goal, and the researcher's measure of adaptation. The probable sources of innovation are specified for each, as well as their likely effects on the measure of interest. It is worth noting that studies may actually report multiple measures of adaptation as defined here. Depending upon the design, it is possible in principle to simultaneously track the retention of features from source material, the actual performance in a given task and the transitioning structural properties of the behaviour being transmitted itself.

• Download figure
• Open in new tab
• Download PowerPoint

(a) Studies measuring loss or distortion of a source

For studies in which participants are presented with a goal of accurate reproduction, variation in retention rates may be found as a result of the ease or difficulty with which this can be achieved. In studies using the serial reproduction method, for example, alternative methods of presenting the stimulus material may facilitate more accurate duplication. Tan & Fay [20], for example, compared the transmission of short narratives under two different conditions. In one condition, participants listened to an audio recording of their predecessors' narration (from recall) of a passage, and then produced their own recording from memory for their successor. In the other condition, participants actually met and interacted with their predecessor in the chain, receiving the account in person in the context of a conversation. Recall was found to be better in the interactive condition. Similarly, Eriksson & Coultas [21] also identified differing retention rates across experimental conditions, finding that narratives were transmitted with higher fidelity when participants received the story from two different individuals, compared with receiving a single individual's reproduction twice.

The effects that these different retention rates have on the measures of cultural adaptation used in the studies is very much in line with the predictions detailed in figure 1, with these studies finding that lower retention generates more rapid loss of detail. This in itself is unsurprising given that between-generation similarity and overall cultural change are effectively being inferred from the same data (i.e. the presence or absence of details from the source material). However, given that we can be relatively confident about the source of innovations in these studies (copying error, as opposed to intentional innovation), these studies also provide an insight into baseline levels of change that should be expected from imperfect transmission alone. This information is useful from the point of view of identifying the role of intentional innovation in other studies.

When participants are given a goal of success on a particular task, rather than a goal of reproduction, it is possible to find variation in retention rates across experimental conditions as a consequence of strategic choice as well as ease of reproduction. However, as detailed in §3 and figure 1, any such strategic shifts ought to have equally predictable effects on measures of loss or distortion of source material.

One example of such an effect comes from Caldwell & Eve's [35] study of participants' designs in a spaghetti tower building task. Participants were encouraged to build their towers to be as tall as possible, in two conditions. In the control condition, participants were told their reward payment was based on the final height of their tower. In the other (‘unpredictable payoff’) condition, participants were told their tower would be subjected to unspecified structural tests before being measured for payment, although in reality, no such tests were carried out. The aim of the experiment was to track the influence of particular tower designs that had been presented to the very first generation of participants, and to determine whether the influence of the seeded designs would persist for longer under conditions of uncertainty about payoffs for novel solutions (in line with a ‘copy when uncertain’ strategy [36]). Members of transmission chains were shown photographs of the towers produced by their two immediate predecessors, which they could choose to copy or not, presumably based on their assessment of the likely utility of this information in relation to the task goal. The overall prediction was supported, with towers in the unpredictable reward condition showing higher between-generation similarity (as evaluated by number of shared features), and evidence of residual similarity to the original seed towers in later generations. This contrasted with the findings from the control condition, in which between-generation similarity was lower, and there was no detectable influence of the seed designs in later generations.

(b) Studies measuring task success

Although the studies discussed above (§4a) offer relatively unsurprising relationships between rates of innovation and rates of change, this relationship definitely appears to be less straightforward in other study designs. In studies tracking measures of success on a particular task presented to participants, variation in retention rates may again arise from strategic shifts in the degree of reliance placed on social versus individual learning, but this may not necessarily translate to different rates of adaptation. In one example of such a study, Caldwell & Millen [7] aimed to build upon previous [15] work, which had identified cumulative improvement in spaghetti tower building over generations of replacement microsocieties, by incorporating an experimental manipulation designed to emphasize the importance of tower stability as well as height. Similarly to [35], participants in the stability-emphasis condition were informed that their tower would be measured following a delay during which structural resilience would be under threat. The resulting uncertainty about the likely effectiveness of different designs appeared to generate a strategic shift towards greater reliance on social information, with towers from this condition being rated as having higher relative within-chain similarity, compared with those built by participants given a straightforward height goal.

The critical question then is how the greater reluctance to innovate impacted on the goal measure of tower height. Interestingly, participants in this condition did not appear to have been placed at a disadvantage in terms of the height of their towers, which did not differ significantly across conditions. Although evidence of cumulative improvement was somewhat clearer in the condition favouring greater innovation (height emphasis only), there was also evidence of height increases over generations in the stability-emphasis condition, in spite of the apparent conformity to particular design types.

Caldwell & Eve [35] followed this up using the seeded-chain design described previously, which explored the persistence of particular designs across two experimental conditions intended to correspond to the predictable and unpredictable contexts from [7]. As already noted, the expectation regarding relative retention rates was supported by examining the retention rate of features from the seeded tower designs, but it was also possible to measure task success in the shape of tower height. Consistent with the earlier findings [7], there was no clear difference between these two conditions in terms of success on the task. In contrast, the specific design used to seed the chains (one of which was superior to the other) had a clear effect on the height of the subsequent towers, common across both of the experimental conditions.

Thus, in both cases, these strategic shifts in the balance between social and individual learning have not been associated with an obvious advantage to greater innovation. This is despite the fact that it must be differences in the likelihood of intentional innovation, rather than the likelihood of error, which account for the differences between conditions. In addition, it is worth noting that in both these studies, the conditions exhibiting lower innovation were ones that in reality needlessly constrained participants' choices (particularly in [35], in which the task was simply framed differently across conditions, and there was no real difference in the way the efficacy of designs was evaluated). In this context, one might expect that there should be a clear advantage to participants in the conditions that simply emphasized maximizing height, without needing to consider trade-offs with probable stability. However, bearing in mind that social learning is critical to the retention of advantageous variants, this may explain why the greater willingness to explore alternatives did not appear to generate benefits at group level, because this necessarily occurred at the expense of the potential for retaining beneficial traits. Overall, these studies certainly provide support for the expectation that innovation rates will not have a straightforward relationship with measures of adaptation focused on task success (figure 1).

(c) Studies measuring presence of a cultural attractor

In a recent study of the cultural evolution of structural simplicity, Kempe et al. [17] compared transmission chains of children with adults, the participants' goal being to reproduce the positioning of random dot patterns on a grid. The hypothesis was that patterns would simplify more in the chains of children, as measured by the clustering of dots and algorithmic complexity. The similarity between adjacent responses could also be assessed, based on the percentage of dots correctly placed on the grid. Thus, it is possible to determine from the data whether greater adaptation was associated with lower levels of similarity. Interestingly, in spite of strong support for the hypothesis that simplification would be stronger in chains of children, there was no difference between the two populations in the between-generation error measures. This suggests that increased rate of simplification was not simply attributable to the children making more errors, so in fact they must have made qualitatively different errors that were more likely to shift responses in the direction of greater structural simplicity. So, although it is not possible to say from these data what effect an increase or decrease in error rate might have had in relation to the rate of adaptation in either adults or children, this clearly demonstrates that similar error rates do not necessarily dictate equivalent rates of adaptation.

Currently, there appears to be very limited evidence of the effects of different rates of retention on measures involving proposed cultural attractors. As noted in §3, there is good reason to believe that higher innovation rates might be associated with more rapid change in the direction of the cultural attractor. However, this remains to be established. Furthermore, it is likely that, as with task success measures, relatively faithful retention may be critical to preserving change in a particular direction, yielding a U-shaped relationship between rates of innovation and adaptation. Further research could clarify the nature of this relationship.

(d) Insights from theoretical models

Theoretical work has suggested that there exists a trade-off between the amount of innovation and the level of adaptation depending on the level of environmental stability. In contrast with the experimental approaches, where the environment (the physical or cultural environment to which the considered cultural trait becomes adapted) is typically held constant, mathematical or computational models can manipulate this freely. Using this approach, it has been shown that asocial and social learning are favoured by natural selection when temporal environmental changes occur in short and long intervals, respectively [8,37–41]. In other words, the faster the adaptive value of a cultural variant is changing, the more advantageous is the individual learning strategy. As individual learning is considered as the innovation mechanism, this result also points to the crucial relationship between environmental stability and the amount of innovation/cultural variation that is needed to adapt to those changing conditions. Innovations (in particular adaptive innovations) provide the basis for social learning to be a successful evolutionary strategy even in changing environments [23,42]. However, owing to the possible adaptive and non-adaptive nature of innovations, there exists an optimal balance between the rate of innovation (expressed by the fraction of the population engaged in individual learning) and environmental uncertainty [42]. The more unstable the environment, the higher the amount of variation needed to ensure efficient adaptation. Naturally, this relationship is greatly influenced by the specific social learning strategy [41,43–47].

It is not obvious how to relate the insights from theoretical models directly to those generated by the experimental studies, but consideration of the reasons for this difficulty highlights constraints and assumptions within the experimental designs. Within the modelling literature, innovations are generally viewed as a means of cultural change and, in particular, a means of tracking environmental change. In contrast, in the experiments reviewed here, the ‘environment’ to which adaptation occurs is either the environment of the mind, or the task plus the mind, and the studies document the process of approaching an equilibrium state from a starting point of either naivety, or from an experimentally induced non-equilibrium state. However, taking this view, the varying effects of innovation rates on cultural adaptation across different experimental designs can perhaps usefully be conceptualized as a consequence of both the shape of the adaptive landscape, and the likelihood of innovations climbing in the direction of local optima. Further research, of both a theoretical and experimental nature, is needed to cross-validate specific conclusions.