Measuring coral reef decline through meta-analyses
Abstract
Coral reef ecosystems are in decline worldwide, owing to a variety of anthropogenic and natural causes. One of the most obvious signals of reef degradation is a reduction in live coral cover. Past and current rates of loss of coral are known for many individual reefs; however, until recently, no large-scale estimate was available. In this paper, we show how meta-analysis can be used to integrate existing small-scale estimates of change in coral and macroalgal cover, derived from in situ surveys of reefs, to generate a robust assessment of long-term patterns of large-scale ecological change. Using a large dataset from Caribbean reefs, we examine the possible biases inherent in meta-analytical studies and the sensitivity of the method to patchiness in data availability. Despite the fact that our meta-analysis included studies that used a variety of sampling methods, the regional estimate of change in coral cover we obtained is similar to that generated by a standardized survey programme that was implemented in 1991 in the Caribbean. We argue that for habitat types that are regularly and reasonably well surveyed in the course of ecological or conservation research, meta-analysis offers a cost-effective and rapid method for generating robust estimates of past and current states.
1. Introduction
The Convention on Biological Diversity, one of the main outcomes of the Rio Summit on environment and development in 1992, called for the protection and sustainable use of biodiversity globally. Ten years later, the signatories to the Convention agreed on a strategic plan to implement, more effectively and coherently, the objectives of the Convention and achieve, by 2010, a significant reduction in the current rate of biodiversity loss at the global, regional and national levels (UNEP 2003).
One of the major obstacles to achieving the latter goal is the paucity of data on trends in the state of natural habitats, particularly at the global scale (Balmford et al. 2003). Jenkins et al. (2003), for example, were able to derive rates of change in area for only four of the nine relatively unmodified natural habitats they considered. Of these, tropical forests, mangroves and sea grass beds have declined globally in areal extent in the past decades, although the data for the latter two habitats are not robust.
There is a clear need for more data to be collected on both the area and state of different natural habitats. Areal data can often obtained through remote sensing, even in some aquatic ecosystems that present significant challenges for remote sensing technology and interpretation. However, whenever degradation precedes structural loss, in situ surveys may be necessary to detect the warning signs of community shifts. For example, while coral reef area can be readily measured remotely, it is not yet possible to derive broad-scale information on coral and algal cover—two major indicators of coral reef health—from remote sensors (Mumby et al. 2004). Unfortunately, global in situ monitoring programmes are expensive and can be difficult to implement and support on a long-term basis. New programmes will also take some years to yield visible trends. Alternatively, better use can be made of information already collected at the local or regional level. For example, a multitude of small-scale ecological studies have been carried out, for practical or academic purposes, in the past decades, in a variety of habitats. These studies often provide data on the extent or composition of the study habitat. Quantitative methods of research synthesis, such as meta-analysis (Hedges & Olkin 1985; Cooper & Hedges 1994), can pull these disparate strands of small-scale information into a larger-scale, longer-term picture of change.
Gardner et al. (2003) recently demonstrated the usefulness of meta-analysis of existing datasets for assessing changes occurring on Caribbean coral reefs. Coral reefs are an important ecosystem, both in terms of biodiversity and for the invaluable goods and services they provide to millions of coastal dwellers at tropical latitudes (Moberg & Folke 1999). Qualitative global assessments of the state of coral reefs exist (e.g. Bryant et al. 1998; Wilkinson 2000), but there is no reliable global estimate of the rate at which coral reefs are being lost (Jenkins et al. 2003). To provide the first quantitative regional estimate of coral loss, Gardner et al. (2003) collated data on live hard coral cover from published and unpublished studies carried out across the Caribbean basin over the past 25 years. Their meta-analysis revealed that coral cover on Caribbean reefs has been lost at an average rate of 5.5% per annum during this period.
In this paper, we examine important issues in the use of meta-analytical techniques to generate trends in the state of natural habitats, using Caribbean coral reefs as an example. We first provide a brief summary of meta-analysis, before showing how it can be applied to quantitatively estimate the ecological changes that have occurred on reefs on a Caribbean-wide scale. We then explore empirically some of the potential biases and sensitivities to data availability of meta-analysis. Since our approach relies on repeated measurements of the same site, which are often not available for all habitats or regions, we examine the affect on estimates of change in live coral and macroalgal cover of constructing time-series from data obtained from different studies in different years at different sites. We also compare estimates of change in coral and macroalgal cover generated by meta-analyses with those obtained from a recent coordinated monitoring programme of Caribbean reefs. Although we focus on coral reefs of the Caribbean basin, the approach used here can be applied to any habitat and region.
2. Meta-analysis in a nutshell
Meta-analysis is a set of methods designed to quantitatively summarize research findings across studies (Hedges & Olkin 1985). The method was developed primarily in medicine and the social sciences, but has, in the last decade, been used extensively in ecological (Côté & Sutherland 1997; Mosqueira et al. 2000; Gurevitch et al. 2001), evolutionary (e.g. Reed & Frankham 2001; Florin & Odeen 2002) and behavioural studies (e.g. Côté & Poulin 1995; Møller & Alatalo 1999; Jennions et al. 2001). Fernandez-Duque & Valeggia (1994) suggested that meta-analysis could also be a valuable tool in conservation research.
A detailed discussion of the statistics of meta-analysis is beyond the scope of this paper, but is provided in Hedges & Olkin (1985) and Cooper & Hedges (1994). The main steps of a generic meta-analysis are summarized here. In meta-analysis, the outcomes of different studies are expressed in terms of a common currency called ‘effect size’. Common measures of effect size are the standardized difference between means of experimental and control groups (d) or the Pearson product moment correlation coefficient (r). However, a wide range of other effect size metrics can be used that may be more appropriate to the hypothesis being tested than d or r (Osenberg et al. 1999). The effect sizes derived from individual studies are then weighted (see below), and combined to yield a common estimate of the magnitude of the effect, bounded by confidence intervals. The homogeneity of effect sizes among studies is examined to determine whether all studies share a common effect size. If there is evidence of heterogeneity, studies can be divided into biologically meaningful subgroups and the overall effect sizes for each subgroup recalculated and compared statistically. This allows the identification of factors that explain significant amounts of variance in effect size (Cooper & Hedges 1994).
Meta-analysis offers at least three important advantages over traditional ‘vote-counting’ methods of research synthesis. In vote-counting, studies are divided into categories according to whether they are statistically significant and whether they support the hypothesis being tested. The proportions of studies ‘voting’ for or against the hypothesis are then counted, and the hypothesis is supported if a significant proportion of studies find in its favour. Because it considers only significant results, vote-counting can be overly conservative (Hedges & Olkin 1985), particularly in ecological studies where sample sizes, and hence statistical power, are often low. Meta-analysis overcomes this problem by considering all studies, regardless of statistical significance, and by weighting the outcome of each study by a correlate of sample size (usually the inverse of the sample variance; Hedges & Olkin 1985). Larger, more robust studies are therefore given more weight in the analysis. A second, related advantage of meta-analysis is that the likelihood of type II errors (i.e. failing to reject the null hypothesis when it is false), which is high for individual studies with low sample sizes or weak treatment effects, is reduced through the amalgamation of all studies into a single analysis. This can be a great advantage for conservation research where committing such an error can be more harmful than committing a type I error (i.e. rejecting the null hypothesis when it is true). In the case of estimating rates of change in a particular habitat, a type II error would lead one to believe wrongly that rates of change are not significantly different from zero, which would have serious consequences for declining habitats or species. Finally, meta-analysis provides a quantitative estimate of the overall magnitude of the effect under study, as well as its statistical significance. In conventional vote-counting, the number of significant findings may bear no relationship to the magnitude of an effect, and vote-counting does not provide information on the latter. This makes meta-analysis a more objective method of synthesis than narrative reviews.
A number of criticisms have been levelled at meta-analysis. These include: (i) the lack of uniformity in studies pooled in meta-analytical reviews, both in terms of methods used and in terms of robustness; (ii) the possibility that the studies included in meta-analyses are not representative of all the studies that have been carried out; and (iii) the potential non-independence of data if, for example, a single study yields multiple effect sizes included in a single meta-analysis. The first issue can be addressed by having clear selection criteria for studies to be included into a meta-analysis (Englund et al. 1999), and by testing specifically for effects of method on effect sizes. There is evidence in ecological studies for the second issue, i.e. a greater likelihood of publishing significant than non-significant results (Jennions & Møller 2002). However, meta-analytical methods allow the assessment of the extent of this problem through the examination of distribution of effect sizes, and the calculation of the number of unpublished, non-significant studies necessary to overturn a significant overall effect size (Rosenberg et al. 2000). Finally, the issue of non-independence of data can be examined by focused or hierarchical tests of homogeneity. We provide empirical examples below of how these issues were examined when assessing rates of change in Caribbean coral reefs.
3. Estimating change in coral and macroalgae on Caribbean reefs
Coral reefs have been extensively monitored in the Caribbean, particularly over the past three decades. Much of this monitoring has been carried out during the course of post-graduate or post-doctoral studies at single locations, although two large-scale coordinated programmes have been in operation since the 1990s. Thus, time-series information on the composition of benthic cover is typically short and spatially focused. The limited spatial and temporal extent of many coral monitoring programmes has drawn much recent criticism (e.g. Connell 1997; Murdoch & Aronson 1999). However, many of these limitations can be overcome using meta-analysis.
(a) Data acquisition and selection criteria
To evaluate the ecological changes occurring on Caribbean reefs, we obtained data on hard coral cover for reefs within the wider Caribbean basin through electronic and manual literature searches, as well as personal communication with reef scientists, site managers and institutional librarians. Electronic literature searches were conducted using the Scientific Citation Index (SCI) and Aquatic Sciences and Fisheries Abstracts (ASFA) from 1981 to 2001, and 1988 to 2001, respectively. All relevant references cited in these publications were also checked. Studies were selected if they reported (i) percentage cover of live hard coral or macroalgae (including fleshy and calcareous species), (ii) with replicated measurements, and (iii) for two or more years from a site within the region. Each sampling area treated as a site by researchers was considered as a separate site in our database. However, when a single site crossed a steep depth contour (e.g. cross-reef transects; Dustan & Halas 1987), transects were separated into groups of similar depth. All values were converted to percentage of total area, if not originally reported in this way. Cover values of less than 1% were rounded up to 1% in each case.
For the purpose of this paper, we omitted the data provided by the Caribbean Coastal Marine Productivity (CARICOMP) monitoring programme, which were included in the overall rate of change in coral cover reported by Gardner et al. (2003). This was done to allow an explicit comparison of the rates of change generated by this programme to those of the meta-analysis of small-scale studies in §7 below. The other difference with the dataset of Gardner et al. (2003) is the addition of two studies from St Lucia and Saba, which together contribute 61 new sites.
(b) Selection of effect size metrics and weighting procedure
The effect size metric chosen must be matched to the question posed (Osenberg et al. 1999). An effect size metric that could be used to examine ecological change over time is the difference in benthic cover between two surveys, d. However, the magnitude of this difference is potentially influenced by the initial value of cover, i.e. sites with high cover in the first survey have greater scope to show decreases in cover, and vice versa. In addition, the magnitude of d may depend on the time elapsed between surveys, which can be highly variable among studies.
Gardner et al. (2003) used as an effect size metric the relative annual rate of change in percentage cover, CR, which was calculated as:
It is important to note, however, that CR is only valid if the change in cover is linear over time. This was the case for coral cover in Gardner et al. (2003). However, with the expanded dataset used here, exponential models of coral cover decline explain variance in yearly coral cover across sites equally as well as linear models (linear: r2=0.57, F1,23=30.44, p<0.0001; exponential: r2=0.58, F1,23=31.76, p<0.0001). In addition, although many time-series of data for individual reefs show linear changes in coral cover over time, the temporal changes in coral and macroalgal cover are nonlinear at many sites. In such cases, CR can be unduly affected by variation in both the length of the monitoring period and by the magnitude (i.e. slope) of the change. In this paper, we thus consider an alternative metric, CRg (the geometric rate of change), to measure annual rate of change:
Although both CR and CRg use only the endpoint cover values of each time-series, CRg probably better represents the constant proportionate change occurring from year to year in exponentially declining or increasing benthic cover components. CRg has the additional and important advantage of yielding the same (logged) rates for symmetric positive and negative changes in cover.
Conventional meta-analyses usually account for within- and between-study sampling errors through weighting means by the inverse of the sum of the sample variance and pooled study variance (Hedges & Olkin 1985). We avoided this procedure because monitoring data, by definition, provide repeated measurements of the same replicates, causing statistical problems related to temporal auto-correlation. However, to give relatively more importance to more robust measures, we weighted each estimate of change by the reef area (in square metres) surveyed to generate the estimate. Survey area was preferred over an alternative weighting such as the number of repeated measurements made since the latter can be misleading (e.g. a few long transects can cover more surface area of reef than many, much shorter ones). Weighting by area surveyed has previously been used in meta-analyses of the impact of marine protected areas on reef fish abundance (Mosqueira et al. 2000; Côté et al. 2001).
Confidence intervals around mean effect sizes were generated by bootstrapping effect sizes for each site and correcting for bias in distribution around the observed mean (Rosenberg et al. 2000). Mean effect sizes are considered significant when the confidence intervals do not include zero. When the data were divided into subsets, effect sizes among categories were compared with the statistic QM (referred to as QB in Hedges & Olkin (1985)) in a manner analogous to an analysis of variance. The significance of QM was tested against a distribution generated from 5000 iterations of a randomization test (Adams et al. 1997; Rosenberg et al. 2000). All meta-analyses were conducted using the software metawin (v.2; Rosenberg et al. 2000).
(c) Rates of change in coral and macroalgal cover
Our search yielded a total of 51 separate studies, which reported coral cover from 294 sites from across the Caribbean (see Gardner et al. 2003 for a distribution map). Of these, 34 studies also reported macroalgal cover for 172 sites across the region. Studies had lasted, on average, 8.3 years (±6.0 s.d.; range: 2–27 years). Information on survey area was given for 251 of the sites reporting coral cover and 158 of the sites reporting macroalgal cover.
Absolute live coral cover across the region has declined from ca. 55% in 1977 to ca. 13% in 2001, an 80% decline (see also Gardner et al. 2003). The overall meta-analysis shows a mean annual decline in coral cover over this period of 4.7%, which is significantly different from zero (
, 95% CI=−6.2 to −3.0, n=251). This figure is slightly, although not significantly, lower than that reported by Gardner et al. (2003) because of the differences in datasets outlined earlier. The rate of change in coral cover generated by an unweighted meta-analysis, which used all sites for which survey area was not available (
, 95% CI=−5.9 to −3.1, n=294), was similar to that of the weighted analysis. The annual rates of loss in coral cover obtained with CRg were greater than those generated with CR, both for the weighted (
, 95% CI=−7.8 to −10.7, n=294) and the unweighted (
, 95% CI=−7.7 to −10.6, n=251) analyses.
In contrast, absolute cover of macroalgae increased between 1977 and 2001, from ca. 2% to ca. 10%, with a peak of 43% in 1987. The overall mean annual rate of change in macroalgal cover was significantly positive (weighted analysis: 
, 95% CI=31.9 to 87.9, n=158; unweighted analysis: 
, 95% CI=30.6 to 79.6, n=172). The annual rates of increase in macroalgae measured with CRg were also significantly positive, but were much smaller and more realistic than those obtained with CR, both for the weighted (
, 95% CI=0.7 to 7.8, n=158) and the unweighted (
, 95% CI=3.6 to 11.8, n=172) analyses.
The discrepancies between the annual rates of change generated by the use of CR and CRg highlight the importance of choosing an appropriate effect size metric. Positive rates of change are more unduly affected by the use of arithmetic means (i.e. CR) than are negative rates of change. For elements of cover that increase over time, CRg is a better measure of rate of change. In the analyses that follow, we consider mainly the behaviour of unweighted CRg.
The figures above point to a dramatic change in the relative importance of the major benthic components of coral reefs that has occurred in the past three decades throughout the Caribbean basin. Coral-to-algal phase shifts have been described for individual Caribbean reefs (e.g. Hughes 1994; Connell 1997; Ostrander et al. 2000), but this analysis provides evidence that these shifts are large-scale phenomena. Gardner et al. (2003) and Côté et al. (unpublished) examined the temporal aspect of these habitat changes and found that the onset of decline of corals and the increase in macroalgae coincided roughly with the mass mortality of the grazing urchin Diadema antillarum in the early 1980s on many reefs (Lessios et al. 1984; Hughes et al. 1987). However, variation among sites in both the timing and intensity of rates of change suggests an important role for the action of alternative triggers of coral decline, such as coral disease (Aronson & Precht 2001) and hurricanes (Woodley et al. 1981).
4. Exploring potential biases of meta-analysis
(a) Non-independence of data
The problem of non-independence of data both within and between studies in meta-analysis is well recognized (Gurevitch & Hedges 1999). Within-study dependence can be generated, for example (i) if studies measure the outcome of experiments in a number of different ways, (ii) if the outcome is measured at several points in time, or (iii) if it is measured at a number of different sites or for a number of different species. Within-study dependence of the first kind can be simply dealt with by carrying out separate meta-analyses on the different measures reported (Gurevitch & Hedges 1999). The second type of within-study non-independence applies to monitoring data since these entail, by definition, repeated measurements of the same replicates. More complex methods are usually necessary to deal with repeated measures (Hedges & Olkin 1985). However, if effect sizes are not calculated for each repeated measurement, but a single effect size characterizes each time-series, then the problem of within-study dependence is not an issue. This is the case, for example, if a rate of change is used as the effect size metric, as with CR or CRg.
Non-independence caused by the inclusion of multiple sites from a single study can potentially generate significant bias in overall effect size calculation. This could occur, for example, if all or most sites monitored in a study are geographically close or ecologically similar to each other, and hence are likely to yield effect sizes of a similar magnitude. Alternatively, monitoring within a study is likely to have been performed using a similar methodology at all sites, and hence effect sizes are more likely to be similar within a study than among studies using different methods (see below).
One can assess the potential bias derived from using multiple sites from the same study in two ways. First, an overall effect size can be calculated using only one site drawn at random from each study. Second, the analysis can be carried out omitting the study (or studies) contributing a disproportionate number of sites to the dataset. Both effect sizes can then be compared with the overall effect size generated using the total dataset. Given the similarity in rates of change of coral and macroalgal cover generated by weighted and unweighted meta-analyses (§3c), we used the latter to explore these potential biases of meta-analysis since it permitted the inclusion of all sites in the analysis.
When we considered only one randomly chosen site from each study, the annual rate of change in coral cover was −8.8% (95% CI=−13.5 to −6.0, n=51), which is not significantly different from that obtained across all sites (figure 1a). For macroalgal cover, inclusion of a single site per study yielded an annual rate of increase of 12.6% (95% CI=4.7–21.1, n=27), which is slightly higher than that obtained from the entire dataset (figure 1b). Similar results for both coral and macroalgal cover were obtained with each of 10 additional replicate subsets of data. Figure 1 Annual rates of change in (a) live coral and (b) macroalgal cover when estimated by meta-analysis using the whole dataset or subsets of the dataset to reduce the occurrence of multiple sites per study. Means are shown ±95% bootstrapped confidence intervals. Sample sizes are given in parentheses.
Two studies contributed a large number of sites to our dataset: the Florida Keys Coral Reef Monitoring Project (FKCRMP; Porter et al. 2002), with 43 sites, and the St Lucia project (C. Schelten, unpublished data) with 50 sites. The removal of the FKCRMP had little effect on the estimates of overall rate of change in coral cover (figure 1a) or macroalgae (figure 1b). Similarly, omitting the St Lucia project did not significantly change the estimated rates of change in coral cover (figure 1a) or macroalgae (figure 1b). The problem of non-independence of data owing to the inclusion of multiple sites per study therefore appears to be negligible in this case.
Non-independence among studies can arise if different studies from the same laboratory, for example, tend to use the same methods, which may generate more similar effect sizes than those obtained from different research groups. In addition, when studies are carried out on different species, one may expect the effect sizes of closely related species to be more similar to each other than to those of more distantly related species (Harvey & Pagel 1991). In such cases, the components of variance in effect sizes can be estimated by nesting studies within laboratories, or species within higher taxonomic levels, and the variance in overall mean effect size can be calculated as a function of these variance components.
(b) Differences in survey methods
One of the strengths of meta-analysis is its ability to combine results from different studies addressing a similar question using a variety of experimental methods. While this may sometimes seem akin to mixing apples and oranges, Rosenthal (1991) argued that generalizing over studies is essentially similar to generalizing over subjects within studies. Nevertheless, meta-analysis allows the direct examination of the effect of using different survey methods on effect sizes.
Four different methods were used in assessing the sessile benthic community on coral reefs in our dataset: line-intercept transects, video-transects (followed by point count analysis), photoquadrats (followed by point count or photogrammetric analysis), and quadrats assessed visually in situ. The biases involved in using some of these methods have been relatively well studied (Done 1977), and the methods are usually remarkably consistent, at least in terms of estimating change in coral cover (e.g. video transects versus chain transects (Rogers & Miller 2001); quadrats versus line-intercept transects (Chiappone & Sullivan 1991); quadrats versus video transects (Garrison et al. 2000)).
For annual rates of change in coral cover, the four commonly used methods yielded similar mean effect sizes (QM3=0.01, p=0.46; figure 2a). However, rates of change in macroalgal cover were significantly higher in studies using photoquadrats and line-intercept transects than the other two methods (QM3=0.44, p=0.001; figure 2b). In the case of line-intercept transects, the large effect sizes were driven mainly by one study carried out in Jamaica, which encompassed a major hurricane impact as well as the demise of the sea urchin D. antillarum (Hughes 1996). Photoquadrats, however, result consistently in high rates of macroalgal increase across the broad geographical scale over which this method was used (i.e. in the US Virgin Islands, Panama, Columbia, Jamaica and Puerto Rico) and across two decades (1977–1998). The annual rate of change in macroalgal cover when studies using photoquadrats are omitted is 3.7% (95% CI=0.4 to 7.1, n=145), which is lower but within the confidence intervals of the overall annual rate of change obtained when all studies are included ( Figure 2 Annual rates of change in (a) live coral and (b) macroalgal cover derived from studies using different survey methods. Means are shown ±95% bootstrapped confidence intervals. Sample sizes are given in parentheses.
, 95% CI=3.6 to 11.8, n=172).
(c) Publication bias
There are few opportunities to measure directly the importance of the ‘file drawer’ (i.e. unpublished studies) problem described in §2. However, this becomes possible when a significant proportion of data is obtained from personal communications or unpublished reports. More than one-third (38%, 19/51) of the studies included in our Caribbean dataset were unpublished. These studies contributed 47% (138/294) of all sites reporting coral cover and 56% (94/172) of sites reporting macroalgal cover. Unpublished studies were of significantly shorter duration than published studies (published: 9.7±6.7 years, unpublished: 5.8±3.8 years; t48.9=2.65, p=0.01); however, several unpublished studies reported benthic cover characteristics on an annual basis for more than a decade.
There was no evidence of publication bias in reports of coral cover. The mean annual rate of change in coral cover from unpublished studies was similar to that of published studies (QM1=0.01, p=0.14; figure 3a). However, unpublished studies yielded a significantly lower rate of increase in macroalgal cover (QM1=0.13, p=0.001; figure 3b). Figure 3 Annual rates of change in (a) live coral and (b) macroalgal cover derived from published and unpublished studies. Means are shown ±95% bootstrapped confidence intervals. Sample sizes are given in parentheses.
There are a number of possible reasons for the discrepancy in rates of macroalgal increase between published and unpublished studies. For example, authors finding small increases in macroalgal cover may be less likely to submit their work for publication, or less likely to be successful in publishing it. More importantly, however, there are methodological differences between published and unpublished studies, with the former using more photoquadrats and fewer visual assessments than unpublished studies (published studies: photoquadrats: 29% of sites, visual assessment: 0% of sites; unpublished studies: photoquadrats: 17% of sites, visual assessment: 40% of sites). In addition, unpublished studies are significantly more recent than published ones. The median start year of unpublished studies was 1995 (interquartile range: 7 years), compared with 1983 (interquartile range: 12.5 years) for published studies (Mann–Whitney U=118.0, N1=32, N2=19, p<0.001). Published studies that began in the early- to mid-1980s captured a period of rapid macroalgal growth, particularly at sites in Panama and Jamaica (Côté et al. unpublished), which may explain why these studies generated significantly higher rates of macroalgal increases than unpublished studies. These published studies have helped create a general perception that coral-to-algal phase shifts are widespread and remain ongoing throughout the Caribbean (McManus & Polsenberg 2004), whereas our data suggest that these shifts have occurred less frequently in the past decade than in the 1980s (Côté et al. unpublished).
5. Sensitivity to sample size and geographical spread of data
The accuracy of meta-analysis in generating estimates of rates of change for any habitat depends on the availability of data. Data availability may be constrained in a number of ways, including the existence of few surveys across the range of the focal habitat, or adequate sampling in only some, but not all, parts of this range.
(a) Sample size
To examine the sensitivity of meta-analysis to sample size, we compared the annual rates of change in coral and macroalgal cover obtained from the dataset used in this paper with those obtained using 20 reduced datasets. Reduced datasets were obtained by selecting at random ca. 20, 25, 33, 50, 67 and 75% of sites available per study (when studies contributed more than one site to the analysis) and by selecting 60–200 sites at random, irrespective of study. To reduce the dataset even more substantially, we randomly selected a single site per study (as in §4a), one site per country using the end points of the longest time-series available, and one site per country using the end points of the shortest time-series available.
The overall rate of change in live coral cover obtained from the full dataset ( Figure 4 Relationship between sample size and the absolute discrepancy in effect sizes between the overall dataset and reduced datasets for (a) live coral cover (regression equation: discrepancy in CRg=−0.0078 × sample size +1.86) and (b) macroalgal cover (discrepancy in CRg=−0.025 × sample size +3.63). The dashed lines indicate the minimum sample sizes required to achieve a maximum discrepancy of 10% of the value of the overall effect size.
) fell within the confidence intervals of all 20 estimates generated from reduced datasets, which had sample sizes ranging from 17 to 261. However, CRg values from reduced datasets differed by up to 2.9% from the overall mean rate of coral decline (i.e. 32% in relative terms), with the discrepancy between values of CRg generated from data subsets, and from the whole set, decreasing with increasing sample size (r2=0.38, F1,18=11.1, p=0.004; figure 4a). Thus, rates of change in coral cover that deviate in absolute terms by less than 0.9% from the whole dataset (i.e. 10% in relative terms) can be obtained with minimum sample sizes of ca. 125 sites.
The meta-analysis of macroalgal cover appeared similarly sensitive to sample size variation. In this case again, substantial decreases in sample size yielded no significant differences in CRg, owing to large variability around these estimates. For macroalgal cover, CRg values deviated by as much as 6.3% from the full dataset (i.e. 80% of the overall mean rate of macroalgal increase). The discrepancy between CRg generated from data subsets and from the whole set decreased significantly with sample size (r2=0.36, F1,18=10.2, p=0.005; figure 4b). Rates of change in macroalgal cover that deviate in absolute terms by less than 0.8% (i.e. 10% of the overall mean rate of macroalgal increase) from the whole dataset can be obtained with minimum sample sizes of ca. 115 sites.
These recommended minimum sample sizes are very likely to be habitat-specific, but they highlight the importance of large sample sizes in meta-analyses to achieve a reasonably good accuracy of estimates. However, this level of accuracy is perhaps unrealistic, particularly for habitat types and regions for which relatively fewer data are available and/or for which no other regional or global data are available. Estimates of change for other biomes derived from coordinated global programmes vary by more than 10% (e.g. the tropical forest loss estimates of FAO (2001) and Achard et al. (2002) vary by 23% ; see Balmford et al. 2003). If a lower level of accuracy was deemed to be informative, then a lower sample size would be required.
(b) Geographical spread
To investigate the effect of uneven geographical representation of data, we assigned sites to one of five geographical areas: northern Caribbean (USA, Bahamas, Bermuda), Greater Antilles (Jamaica, Puerto Rico, Cuba, Dominican Republic, and owing to geographical proximity, the Cayman Islands), Lesser Antilles (US Virgin Islands, Barbados, St Lucia, Saba), northern South America (Curacao, Bonaire, Venezuela, Colombia and Tobago) and Central America (Mexico, Belize, Honduras, Costa Rica, Panama and Nicaragua). We then compared the annual rates of change in coral and macroalgal cover obtained with the whole dataset and when omitting each geographical area in turn.
Estimates of change in live coral cover were not significantly different from the overall rate of change when the northern Caribbean, Greater Antilles, Lesser Antilles, northern South America or Central America were each omitted separately (all QM1<0.008, p>0.19; figure 5a). These rates of coral decline deviated from the overall rate by 0.1–1.6% in absolute terms (1–17% in relative terms). The rates of change in macroalgae, when omitting separately each of the five subregions, were also similar to the overall rate of change (all QM1<0.006, p>0.49; figure 5b). These rates of increase deviated from the overall rate by 0.1–2.3% in absolute terms (1–30% in relative terms). For both coral and macroalgae, omission of the Northern Caribbean caused the largest discrepancies. This subregion contributed the second largest proportion of sites to the dataset (81 or 28% of sites), suggesting that the discrepancies may be in part caused by sample size reduction. However, some subregional variation in rates of change probably also exists, making these meta-analyses slightly sensitive to geographical patchiness in data coverage. Figure 5 Annual rates of change in (a) live coral and (b) macroalgal cover derived from the overall dataset and by omission of each of five subregions. Means are shown ±95% bootstrapped confidence intervals. Sample sizes are given in parentheses.
6. Alternative to time-series of data
Estimates of change in any habitat should ideally be derived from surveys carried out repeatedly in exactly the same location to control for small-scale environmental heterogeneity. In practice, however, single surveys probably vastly outnumber repeat surveys. Can these snapshots of habitat composition nevertheless be incorporated into meta-analyses and contribute to measurements of environmental change?
To test this possibility, we divided sites within each country such that one half of sites contributed their start percentage cover and the other half contributed their end cover. The mean start and end covers were then calculated, with these mean values being derived from a different set of sites in each country. The median start and end years were obtained for each start and end group, and used to calculate annual rates of change in coral cover for each of the 18 countries or islands in the database.
The rate of decline in coral cover obtained from the country-level randomly matched estimates was 9.0% per annum (95% CI=−13.0 to −5.6, n=18), which is not significantly different from the rate of coral decline detected in the original dataset (QM1=0.0012, p=0.95). Transposing the sites between groups (i.e. sites contributing start percentage cover now contribute end cover, and vice versa) yielded similar results (QM1=0.002, p=0.52).
These results suggest that snapshot surveys could be useful in generating reasonably accurate rates of environmental change. However, this accuracy will depend on the availability of reasonably large numbers of estimates for discrete time periods, which can be combined over a relatively broad but biologically or politically relevant geographical scale (e.g. state, country).
7. Meta-analysed estimates of change versus other estimates
How do meta-analysed estimates of change in coral and macroalgal cover compare with rates obtained from coordinated monitoring programmes?
One ongoing programme of reef monitoring in the Caribbean region is CARICOMP (UNESCO 1998). CARICOMP is an international monitoring network which began in 1990 with funding from UNESCO. The network comprises 27 institutions in 17 countries across the wider Caribbean, which are involved to varying degrees in the monitoring activities. Fully participating institutions undertake yearly ecosystem measurements using a standardized methodology on relatively undisturbed coral reefs (one to four reefs per country) and in other coastal habitats, along with oceanographic and meteorological measurements. Transect sites are permanently marked to allow repeated surveys of the same locations. The ‘best’ reef zones are specifically chosen as transect sites (i.e. areas with the most Monstastrea annularis coral or, if this species is rare, areas with high coral diversity; CARICOMP 2001). Most CARICOMP data currently available cover the period from 1993 to 2001.
The rate of decline in coral cover at CARICOMP sites is similar to that obtained from our dataset for a comparable time period (1993–2000; QM1=0.005, p=0.78; figure 6). The rate of increase in macroalgal cover, however, is one order of magnitude lower in the CARICOMP programme than for our meta-analysed data (figure 6). This difference is not significant, owing to the large confidence intervals around the CARICOMP mean (QM1=0.03, p=0.16). In fact, the rate of macroalgal increase is not significant at CARICOMP sites (i.e. the CI overlaps zero). This is probably a consequence of the non-random selection of sites within the CARICOMP programme, as well as the fact that CARICOMP has been active only in the 1990s, when rates of macroalgal increases in the Caribbean had slowed down considerably compared with the previous decade (Côté et al. unpublished). Figure 6 Annual rates of change in live coral cover derived from the CARICOMP monitoring programme, and from the meta-analyses of small-scale studies (i.e. this study) carried out for the equivalent time span. Means are shown ±95% bootstrapped confidence intervals. Sample sizes are given in parentheses.
8. Conclusions
Live coral cover throughout the Caribbean has declined significantly for the past three decades. This decline has been accompanied by a great increase in macroalgal cover. These patterns had previously been described qualitatively (e.g. Wilkinson 2000), but a meta-analytical synthesis of data derived from mostly short-term and small-scale surveys allowed the quantitative assessment of the rates of change in these two key benthic components of coral reefs on a Caribbean-wide scale (Gardner et al. 2003; Côté et al. unpublished). Standing at 9.2% per annum over the past 25 years, the rate of decline of coral appears to exceed those reported for most other habitats (Balmford et al. 2003; Jenkins et al. 2003). However, unlike the latter rates, which usually indicate reductions in habitat area, the former reflects instead a thinning-out of the main contributors to growth of the reef framework, which will ultimately lead to structural decline and loss of extent (e.g. Lewis 2002). In ecosystems such as coral reefs, where ecological shifts are a precursor to physical loss, it is therefore imperative to assess the rate at which degradation occurs.
The rate of change in coral cover we derived through meta-analysis of disparate studies is similar to that obtained with a coordinated monitoring programme using standardized methodology (i.e. the CARICOMP programme) for the same time period (1993–2000). Meta-analysis, however, allowed us to examine the pattern of ecological change that preceded the first CARICOMP surveys by nearly two decades, providing a better context for the rates of coral decline witnessed in the Caribbean today. We believe that meta-analysis could be similarly used to generate estimates of change in cover of live coral and other benthic components for other regions of the world and, indeed, rates of change for any habitat that is regularly and reasonably well surveyed in the course of ecological or conservation research.
In this paper, we explored some of the potential biases and sensitivities of meta-analysis. For example, non-independence of data arising from the inclusion of several sites per study was not a major issue in our analyses. This may not be a general conclusion but it is relatively easy to verify the extent of this problem using the methods presented here. The inclusion of data generated with different methods can also generate bias, but again the magnitude of this effect can be measured within the meta-analytical framework. Methods that are found to generate significant heterogeneity in the overall effect sizes can either be excluded, or considered in separate analyses.
Publication bias is likely to affect many datasets (Jennions & Møller 2002). Although we found no evidence of an effect of publication bias for data on live coral cover, this was not so for macroalgal cover. In the latter case, exclusive use of published data could have resulted in significant biases. This suggests that considerable effort must be made to include all data when carrying out meta-analyses. It is revealing that more than one-third (ca. 38%) of the data used in our analyses was obtained from grey literature (e.g. unpublished reports or dissertations) or from personal communications. A wealth of information clearly exists, but may not be readily available without determination on the part of those assembling datasets and openness by those owning the data. There are established central repositories of raw data (e.g. the UNEP World Conservation Monitoring Centre), but these will remain of limited usefulness until researchers routinely contribute to their holdings.
All meta-analyses are sensitive to data availability. We found here that the accuracy of the estimated rates of change in both coral and macroalgal cover depended on sample size. The recommendations on minimum sample sizes (100+ sites in each case) made here to achieve a reasonable accuracy (i.e. 10% ‘error’) are almost certainly habitat-specific and probably even region-specific. The geographical distribution of these sites will also be important. In our analyses, omitting part of the range under study did not lead to significant differences from the overall rates of change in coral or macroalgal cover. However, the sensitivity of meta-analysis to geographical distribution of data will depend on the existence of significant spatial variation in rates of ecological change, which will of course be difficult to establish in the absence of data from parts of the range under study. In such situations, data coverage should be as complete as possible and more effort should be directed at locating unpublished information from areas that appear to be data-poor. The identification of gaps in empirical data can also guide future surveying efforts.
Our analyses show that one-off surveys can potentially contribute to our understanding of large-scale patterns of change. Snapshot surveys may generally need to be more numerous than repeat surveys to provide an accurate picture of change. However, increasing numbers of one-off surveys are currently being carried out, using standardized methodology, by volunteers participating in programmes such as Reef Check (Hodgson & Liebeler 2002). Such programmes may therefore be an important source of data in the future, as well as fulfilling a crucial role in public awareness and education.
It is clear that meta-analysis offers a powerful tool to quantitatively estimate current rates of ecological change in a variety of habitats and to reconstruct recent patterns of change. It offers two particularly significant advantages for conservation. First, by using existing data collected for a variety of purposes, meta-analysis removes the need to wait for coordinated monitoring programmes to generate evidence of ecosystem stress. It should nevertheless be stressed that such monitoring programmes remain the best sources of information about environmental change. Second, for coral reefs, and perhaps other habitats, monitoring and evaluation are not usually specifically linked to the identification of stressors, which creates an obstacle to transposing scientific information into policy and legislative frameworks (Risk 1999). Meta-analysis allows an examination of the causes of change (e.g. Gardner et al. (in press) on the effects of hurricanes on rates of coral decline), producing results that can feed directly into management and mitigation programmes. Creating a direct link between science and action is the only way to achieve the goals set by the Convention on Biological Diversity.
We are forever grateful to Richard Aronson, Rolf Bak, John Bythell, Don Catanzaro, Leandra Cho, Peter Edmunds, Ginger Garrison, Francisco Geraldes, Catriona Glendinning, Keith Hackett, Alastair Harborne, Edwin Hernandez-Delgado, Zandy Hillis-Starr, Walt Jaap, Lisa Kellogg, Tim McClanahan, Melanie McField, Jeff Miller, Thad Murdoch, Richard Nemeth, William Precht, Caroline Rogers and Gene Shinn for contributing unpublished data or manuscripts in press. Thank you also to Andrew Balmford for giving us the opportunity to write this paper and for his insightful comments, and to David Hutchinson for assistance with data analysis.
Glossary
| ASFA | Aquatic Sciences and Fisheries Abstracts |
| CARICOMP | Caribbean Coastal Marine Productivity |
| FKCRMP | Florida Keys Coral Reef Monitoring Project |
| SCI | Scientific Citation Index |
| UNEP | United Nations Environment Programme |
| UNESCO | United Nations Educational, Scientific and Cultural Organization |
Footnotes
One contribution of 19 to a Discussion Meeting Issue ‘Beyond extinction rates: monitoring wild nature for the 2010 target’.
