Ancient whales did not filter feed with their teeth

The origin of baleen whales (Mysticeti), the largest animals on Earth, is closely tied to their signature filter-feeding strategy. Unlike their modern relatives, archaic whales possessed a well-developed, heterodont adult dentition. How these teeth were used, and what role their function and subsequent loss played in the emergence of filter feeding, is an enduring mystery. In particular, it has been suggested that elaborate tooth crowns may have enabled stem mysticetes to filter with their postcanine teeth in a manner analogous to living crabeater and leopard seals, thereby facilitating the transition to baleen-assisted filtering. Here we show that the teeth of archaic mysticetes are as sharp as those of terrestrial carnivorans, raptorial pinnipeds and archaeocetes, and thus were capable of capturing and processing prey. By contrast, the postcanine teeth of leopard and crabeater seals are markedly blunter, and clearly unsuited to raptorial feeding. Our results suggest that mysticetes never passed through a tooth-based filtration phase, and that the use of teeth and baleen in early whales was not functionally connected. Continued selection for tooth sharpness in archaic mysticetes is best explained by a feeding strategy that included both biting and suction, similar to that of most living pinnipeds and, probably, early toothed whales (Odontoceti).


Specimen selection
We analysed the cheek teeth of five modern pinnipeds, four terrestrial carnivorans, and eight extinct cetaceans (figure S1; supplementary table S1). All of the living species have known feeding styles, and thus serve to create a morphofunctional framework within which the fossils can be placed. Only unworn or minimally worn teeth were used for the analysis, including a newly referred specimens of Janjucetus and Coronodon (see below). For each pinniped, we selected one lower third postcanine (middle of the tooth row). For the terrestrial carnivorans, we chose the lower fourth premolar, thereby avoiding the specialised carnassials which have no direct equivalent in marine mammals. For the fossil cetaceans, we were limited by the sparseness and quality of the available material, and consequently chose whichever teeth were available and best preserved.

Scanning
We scanned original teeth for all living species, but used high-resolution casts for the fossils (owing to transport restrictions, and to minimise damage). Specimens were scanned using either a 3D surface laser scanner or micro-computed tomography (microCT) (supplementary table 1). Laser scanning was carried out using a Laser Design DS-Series 2025 3D scanner with a RPS-120 laser probe (620 nm) (Laser Design Inc., Minneapolis, MN). Original specimens were coated with ammonium chloride (NH4Cl) to prevent the laser from passing through the translucent enamel. Figure S1. Overview of all of the teeth included in this study. Not to scale.
Laser scans were made using a point spacing of 10 μm with repeated scan passes over the specimen at 60 degree increments. Repeated data were removed using a 0.005 μm point spacing uniformity filter. As the scanner uses line of sight to reconstruct the model, it was sometimes difficult to record surface details within deep intercusp notches (especially in Lobodon). To scan these regions, we moulded the specimens in question using high-resolution silicon (light fast-set vinyl polysiloxane precision impression material, Provil novo), and then cut and scanned these moulds directly to generate a model of the previously invisible surfaces. The point clouds resulting from each scan were assembled into a 3D model (.ply file format) in Geomagic 12 (Geomagic Inc., North Carolina, USA).
MicroCT scans were carried out using either a Skyscan 1174 microCT scanner (Bruker MicroCT, Belgium) with a source voltage of 50kV, a current of 800 uA, a pixel size of 13.1 μm, and a 0.5 mm aluminium filter; or a Zeiss Xradia 520 Versa (Oberkochen, Germany) at the Monash University X-ray Microscopy Facility for Imaging Geomaterials (XMFIG), with a source voltage of 140 kV, a current of 70 μA, an exposure time of 2 seconds per image, and a pixel size of 12.7 μm. 3D surface models were generated in Avizo v9.0.0 (Visualization Science Group).
In Fucaia, Hydrurga (NMV C7378), Llanocetus, Panthera leo and Zygorhiza, minor wear on the tip of the main cusp was conservatively accounted for using the "curvature" setting of the fill-holes function of Geomagic 12, which provides a reconstruction based on the curvature of the surrounding unworn surface mesh. For Coronodon, minor distortion of the mould required adjusting the 3D model by digitally removing a slice that equalled the flashing in width, and reconstructing a portion of the intercusp notches (figure S2). Reconstructions were conservative and underestimate actual sharpness.

Measurements
We analysed the sharpness of ten different functional regions on each tooth. These included: (i) the anterior, posterior, lingual, and labial sharpness of the main cusp (measured at 50% cusp height; see below and figure S3); (ii) the sagittal and transverse sharpness of the tip of the main cusp (figure S4); and (iii) the sagittal, transverse, anterior horizontal, and posterior horizontal sharpness of the first posterior intercusp notch (figure S5). For Coronodon sp. only, the first anterior intercusp notch was measured instead, owing to damage to the posterior notch. Both notches appear extremely similar in sharpness and thus should yield comparable measurements. Sharpness was measured as follows: Step 1: Each 3D model was imported into Rhino 5 (McNeel North America, USA) and oriented so that the long axis of the primary cusp was parallel to the vertical axis in 3D space (z), and the sagittal axis parallel to the x-axis (figure S3).
Step 2: The height of the main cusp was measured along the vertical (z) axis from its tip to the base of the first posterior inter-cusp notch (figure S3). At 50% of this height, we first generated a horizontal cross section through the cusp, and then calculated the square root of its 2D area. Finally, we used this value to calculate a standardised length (StandardLength) for each tooth: Step 3: At each functional region, we generated a 2D cross-section -either along the orientation of the main cusp or along the orientation of the first posterior intercusp notch (figures S3-S5) -and ruled a line from its sharpest (or distalmost) point inward. After a distance equal to StandardLength, this line was capped by a second, perpendicular line that spanned the cross section and delineated an area immediately inside its sharpest point (figure S3). The relative size of this area is an indicator of the relative sharpness of the functional region, with larger areas being blunter.

Analysis
Analysis of the data was carried out in R, version 3.2.4 [1], using the packages vegan [2], ggplot2 [3] and MASS [4]. To enable comparisons across teeth irrespective of overall size, we scaled all ten sharpness measurements as follows (see supplementary table S2 for raw measurements): Scaled Measurement = (SQRT(SharpnessAreaMeasurment)/SQRT(CuspArea)*100 We subjected all measurements to a Principle Components Analysis (PCA), to determine whether the fossil cetaceans clustered with any of the living species with known feeding styles. In addition, we performed a Discriminant Function Analysis (DFA) of the extant species only, which were grouped according to the use of the postcanine teeth for either filtering (Lobodon and Hydrurga) or raptorial feeding (all remaining seals and terrestrial carnivores), respectively. Finally, we used the results of the DFA to classify the fossils cetaceans in terms of their likely feeding capabilities. Where multiple individuals were measured for a species, these were averaged before being included in the multivariate analyses as a single data point.

Supplementary discussion of Coronodon
The basal mysticete Coronodon from the Early Oligocene of South Carolina, USA, is known from a nearly complete skull with a well-preserved tooth row [5]. Coronodon is the latest extinct mysticete to be interpreted as a tooth-based filter feeder, based on a series of detailed arguments regarding its dental morphology and tooth wear [5]. In brief, the idea of filtering in this species is based on (i) the presence of large, highly emergent and notably denticulate postcanine teeth; (ii) broadly overlapping upper and lower tooth rows; (iii) the presence of radially oriented accessory denticles, thought to facilitate prey retention; and (iv) evidence for water flow through the putative filtering slots, based on the local absence of caries-related dental erosion.
Overall, Coronodon has been interpreted as capable of both raptorial and tooth-based filter feeding, similar to extant leopard seals (Hydrurga leptonyx). Unlike previous hypotheses about tooth-based filtering in cetaceans [6], which emphasised the elaborate morphology of individual teeth, feeding in Coronodon envisages two different types of filtration: one in which the lower jaw is abducted, and filtrations occurs via large (15 x 35 mm), diamond-shaped gaps between the upper and lower tooth rows; and one in which the mouth is (nearly) closed, and prey is filtered through narrow (0.5-3 mm), denticle-rimmed slots between the imbricated lower teeth.
Here we discuss the feeding strategy of Coronodon in light of our results, and with specific reference to the six points listed in the main text. We conclude our discussion with an alternative interpretation, which, we suggest, better fits the available evidence.

Dental morphology
Our quantification of tooth sharpness provides a chance to test the 'interdental' filter feeding hypothesis for Coronodon. Filtering through gaps and slots between teeth could theoretically be envisaged without specific adaptations to tooth crown morphology. Nevertheless, water still has to pass the denticles and notches framing each gap, with the denticles themselves thought to maximise prey retention [5]. A similar situation exists in leopard and crabeater seals, where the tooth filter consist of highly elaborate teeth held in occlusion [7,8]. Even interdental filtration should thus benefit from adaptations facilitating water flow, and plausibly result in a measurable change in dental morphology.
Our analyses unequivocally cluster Coronodon with terrestrial carnivorans, non-filtering pinnipeds and other toothed mysticetes (figure 2). Coronodon retains sharp cutting edges, suggesting continued selection for sharpness. This is consistent with the presence of caniniform incisors and abrasion of the right P2 in the holotype [5], and suggests that the teeth continued to be used for prey processing. At the same time, there are no obvious adaptations that could facilitate water flow, and thus no evidence in support of filtering.

Stable isotopes
Extant mysticetes forage at a lower trophic level than odontocetes, thanks to their filter feeding strategy and generally small-sized prey. Trophic fractionation means that this difference is reflected in the stable carbon isotope composition of their teeth and bones, with carbon isotope values for mysticetes being significantly lower than those for odontocetes [9]. A similar pattern has been observed in fossils, with extinct mysticetes likewise showing significantly lower carbon values than their odontocete kin [9].
A previous isotope study [9] sampled both a juvenile individual potentially conspecific with Coronodon havensteini (ChM PV4745) [5: e1], and a specimen recovered as sister to C. havensteini (ChM 5720) [5: fig. 4]. Both specimens showed elevated carbon values closely resembling those of a fossil odontocete from the same locality [9: fig. 9A], suggesting that they fed at a high trophic level. Feeding high in the food chain correlates with relatively large prey, and is thus inconsistent with filteringin particular, with regard to the extremely small particles that could have been trapped by the interdental slots (see also section 2.4 below). Feeding on small forage fish, rather than crustaceans, is unlikely to explain this pattern, as small fish also form part of the diet of several extant rorquals [10], and at least some fossil mysticetes [11]. Likewise, a mixed dietcomposed of large prey items taken raptorially, and small items filtered from the waterwould presumably result in a carbon isotope signal intermediate between that of odontocetes and toothless mysticetes.

Radially oriented denticles
Unlike archaeocetes, aetiocetids and mammalodontids, Coronodon and a variety of other archaic mysticetes possess radially oriented accessory denticles thought to enhance prey retention during filtering [5]. However, radial denticles also occur in the archaic mysticete Mystacodon [12] (not included in the phylogeny of Geisler et al. [5]). The latter shows an extreme degree of dental wear that would have prevented the teeth from being an effective filter, and thus contradicts the association of radially oriented denticles with filtration. Radial denticles are furthermore absent in filter feeding seals (contra [5]), with the denticles of both leopard and crabeater seals curving inwards towards the main cusp ( figure S6). Figure S6. Dentitions of (a) the leopard seals, Hydrurga leptonyx (NMV C13866), and (b) the crabeater seal, Lobodon carcinophaga (NMV C7385). Note the vertically oriented, inwards curving denticles on the postcanines. Not to scale.

Tooth wear in Coronodon
The tooth crowns of Coronodon are marked by a variable degree of apical abrasion. Crucially, such wear also occurs on the basal, radially directed denticles surrounding the interdental slots. These denticles plausibly ought to have been sheltered from strong abrasive forces, and the wear on them has hence been interpreted as resulting from prey and/or sediment particles accumulating at the interdental slots during filtering [5].
If sediment particles flushing past the teeth abraded the basal denticles, then similar wear presumably should be present all along the (internal) rim of the interdental slot. We agree that abrasion may be more pronounced on the relatively exposed tips of individual denticles, but it seems unlikely that sediment capable of damaging the enamel would not also affect the remainder of the slot. In line with this view, abrasion tends to affect a large portion of the crown in species where sediment causes obvious wear [13][14][15].
Apical abrasion of the basal denticles by trapped prey is plausible, but difficult to achieve given the hardness of enamel and the lack of a strong force pressing prey into the denticles during filtering. Thus, tooth-filter feeding pinnipeds generally do not show abrasive wear on their postcanines [7,16]. Wear as a result of filtration is made even more unlikely by the high carbon isotope values of Coronodon and its close relatives (section 2.2), which point to prey items too large to accumulate next to individual slots.
We suggest that minor abrasion of the basal denticles could be due to biting, despite their radial orientation. As the jaws closed, large prey items would have been compressed and squashed between the teeth, thereby moving vertically and horizontally past most of the crown. The force applied to the radially oriented basal denticles would have been low, but over time could still have damaged their exposed tips. In addition, the mesially oriented denticles, in particular, could have been abraded by large food items passing along the internal face of the tooth row, e.g. during suction feeding, which may explain their somewhat greater degree of wear.

Water expulsion
Abrasion of basal denticles, and the absence of dental erosion near the base of the crown, have been hypothesised to mark the path of water flowing out of the oral cavity during filtering [5]. Irrespective of the problems with interpreting apical abrasion in this way (section 2.5), we agree that water expelled from the oral cavity most likely passed between the teeth. However, water expulsion, whether through the teeth or not, is not per se an indicator of filtering. In discussing this point, it is necessary to recall two principles: (i) Dealing with excess water ingested during prey capture is a challenge faced by all aquatically feeding mammals [17]. Often, such water is actively expelled, as has been established through behavioural experiments and observations in the wild [18][19][20][21]. Some form of water expulsion follows all aquatic feeding strategies, including raptorial, suction and filter feeding, and by itself does not constitute evidence for any of them.
(ii) Filtering is a particular behaviour that can and, for the purpose of discussion and further investigation, should be unambiguously defined. We here follow [17: table 1] in defining filtering as "separation of small food items from water using a dedicated filtering structure, such as specialized teeth or baleen". The emphasis here is on the presence of a dedicated filter, i.e. a specialised structure that improves the ability of a species to retain small prey. The function of baleen in this regard in undisputed [22], and our results (figure 2) confirm quantitatively the longstanding idea that specific filtering adaptations have also shaped the teeth of crabeater and leopard seals [7,8]. It is tempting to view other, simpler forms of separating prey from waterfor example, retaining a single, or even a few, captured fish behind a row of non-specialised teethas filtering. However, this interpretation is misleading: if simply passing water through the teeth is sufficient, then pilot whalesknown to feed via combination of ram and suction, followed by active water expulsion [18] should also be classified as capable of filter feeding, as should a variety of other odontocetes and pinnipeds (e.g. Australian and subantarctic fur seals [20]). While perhaps defensible from a strictly behavioural point of view, lumping the simple sieving behaviour of most cetaceans and pinnipeds with true filtration would paint aquatic mammal feeding strategies with too broad a brush. Ultimately, it would ignore the fundamental transition that chaeomysticetes and filter-feeding seals underwent when they lost (or transformed) the primary food capture tool of vertebrates, and replaced it with a dedicated filter serving an (at least for mammals) unprecedented purpose.
In light of these principles, how can the evidence for oral water flow in Coronodon be interpreted? Given that tooth sharpness and notch shape do not reveal any specific adaptations for filtering (section 2.1), and that radially oriented denticles occur in non-filtering species (section 2.4), there seems to be little evidence for the presence of a dedicated filter. Filtering would furthermore have been of little benefit to Coronodon, given that stable isotopes speak against its targeting small prey (section 2.2). Overall, the most parsimonious interpretation of water flow out of the oral cavity is therefore water expulsion following raptorial and/or suction feeding.

An alternative interpretation of feeding in Coronodon
Given the problems with the tooth filtering hypothesis, we offer an alternative interpretation that accounts for the morphological specialisations of Coronodon: suction feeding. First, despite its relatively low mandibular bluntness index, the rostral margins of Coronodon are straight, leading to a noticeable increase in rostral surface area relative to basilosaurids [5: figure 1B,C]; a broader rostrum results in a larger oral cavity, and thus would have facilitated the generation of suction [23]. Secondly, instead of serving as a filter, the highly emergent tooth rows and thick gums could have helped to create a more efficient suction opening by closing the lateral gape [23]. Finally, an increased reliance on suction, rather than raptorial feeding, may account for the relatively thinner enamel of Coronodon relative to archaeocetes [5].
Our interpretation is not precluded by the absence of tooth wear indicative of suction, such as pronounced horizontal striations [15]. Abrasion is dependent on the nature of the ingested particles, and best developed in species that feed mostly benthically [13,24]. Species foraging in a pelagic setting may lack obvious suction-related wear, and even when sediment is ingested, abrasion is by no means guaranteed. For example, crabeater seals have been found with sand in the stomach, yet generally lack abrasion on the postcanines [16].
Overall, Coronodon likely employed a mix of raptorial and suction feeding. Its large gape [5] would have enabled it to capture and process prey raptorially, thus explaining the presence of apical wear and continued selection for sharp teeth. On the other hand, partial abduction of the lower jaw with the tooth rows broadly overlapping would have been employed during suction feeding. There is not enough information to gauge the relative importance of raptorial vs suction feeding: both strategies are consistent with the isotopic data [9], and both are reflected in the morphology of the skull. Both could furthermore have worked together, as seen, for example, in Australian fur seals [19].
Although we disagree that Coronodon filter fed with its teeth, its anatomy fits well with the bigger picture of mysticete evolution. Coronodon represents the fourth lineage of archaic mysticete showing adaptations for suction (besides mammalodontids [13], aetiocetids [15] and, possibly, Mystacodon [12]), supporting the idea that suction feeding may have preceded the evolution of baleen [15,17]. Enlarged gums similar to those of Coronodon likely occurred in other archaic mysticetes [13,15], and may foreshadow the evolution of baleen in chaeomysticetes.  Locality and horizon. Between Bird Rock and Fishermans Steps, Jan Juc, central coastal Victoria, Australia. The specimen was collected in 2013 by P. Mullaly from a loose boulder of light grey glauconitic marl derived from the Jan Juc Marl. Details of the exact locality are available directly from Museums Victoria. The Jan Juc Marl exposed in the coastal section between Bird Rock and Fishermans Steps was deposited between the late Oligocene and earliest Miocene (early Chattian to earliest Aquitanian), approximately 28.10-22.82 Ma (see [25] and references therein).

Diagnosis of NMV P252376
Diagnosis. Differs from the teeth of all aetiocetids (except Morawanocetus) by having labial enamel ornament; from Llanocetus by much smaller size, no broadly palmate denticles, and lacking an interradicular space between basal portion of the roots; from the lower cheek teeth of Morawanocetus by being higher crowned and lacking a distinct entocingulum; and from the lower cheek teeth of Mammalodon by lacking a shelf-like cingulum around the posterolingual corner of the crown base.
Similar to the lower cheek teeth of Janjucetus hunderi by having similar crown length and height; well-developed labial and lingual enamel ornament; prominent labial inflation of the crown at the level (anteroposteriorly) of the anterior edge of the main denticle; and four posterior denticles (posterior denticle 4 is vestigial). NMV P252376 has some features not preserved on the type lower cheek teeth of Janjucetus hunderi owing to wear of the latter, including a sharp anterior carina and three anterior denticles.