Abstract
Nearly all animals forage to acquire energy for survival through efficient search and resource harvesting. Patch exploitation is a canonical foraging behaviour, but there is a need for more tractable and understandable mathematical models describing how foragers deal with uncertainty. To provide such a treatment, we develop a normative theory of patch foraging decisions, proposing mechanisms by which foraging behaviours emerge in the face of uncertainty. Our model foragers statistically and sequentially infer patch resource yields using Bayesian updating based on their resource encounter history. A decision to leave a patch is triggered when the certainty of the patch type or the estimated yield of the patch falls below a threshold. The time scale over which uncertainty in resource availability persists strongly impacts behavioural variables like patch residence times and decision rules determining patch departures. When patch depletion is slow, as in habitat selection, departures are characterized by a reduction of uncertainty, suggesting that the forager resides in a lowyielding patch. Uncertainty leads patchexploiting foragers to overharvest (underharvest) patches with initially low (high) resource yields in comparison with predictions of the marginal value theorem. These results extend optimal foraging theory and motivate a variety of behavioural experiments investigating patch foraging behaviour.
1. Introduction
Foraging is performed by many different species [1–5] and engages cognitive computations such as learning of resource distributions across spatiotemporal scales, route planning and decisionmaking [6]. Comparing species, one can ask how these integrated processes have been shaped by natural selection to optimize returns in the face of environmental and physiological constraints [6,7]. Foraging thus provides the opportunity to study and quantify how both evolution and neural circuitry shape a natural behaviour [8–11].
In natural landscapes, foraging involves a decision hierarchy that unfolds across multiple length and time scales, which consider both where to forage as well as how long to exploit a certain resource [12,13]. On long time scales, animals accumulate evidence to choose which of a collection of large areas they will dwell in and forage, during which their activity does not appreciably change the resource landscape. Following previous work, we refer to this as ‘habitat choice’ [12]. Within habitats, animals exploit resources on shorter time scales, while their activity depletes resources in visits to localized regions. We refer to this behaviour as ‘patch exploitation’ or ‘patch leaving’. Questions of where to forage and how long to exploit local patches of resource constitute a multilevel framework for examining behaviour across spatial scales; still larger scales consider the home range of an individual, as well as the species range [13,14]. For both habitat choice and patch leaving, the local regions of an environment can be conceptualized as a ‘patch’; the key difference is in whether or not the forager’s activity impacts the resource availability in the landscape. As a decision problem, both are sequential choice processes, so the forager does not make a choice between discrete alternatives that are presented simultaneously, but rather only receives evidence from the current patch and must decide whether to stay or go [15]. Thus, habitat choice and patch leaving are related but differ in the length and time scales involved (figure 1a). However, most theoretical work has considered these two problems separately.
Regarding habitat choice, the optimal behaviour is clear: the forager should locate and spend as much time as possible in the habitat patch that maximizes fitness outcomes. Observational studies of habitat choice often consider the combined effects of multiple factors to ask how well they predict observed habitat use (e.g. [16]). Theoretical and experimental studies have investigated multiple factors, such as densitydependent effects predicted by the ideal free distribution when there are multiple foragers on a landscape [17] and how perceptual constraints may lead to deviations from optimal choices [18]. Our focus here is specifically on understanding an accumulation process by which individuals use information to reach decisions on habitat choice, and deriving this result in a mathematically tractable model that can more precisely parameterize strategy and performance in a wide range of environments.
Patch leaving considers shorter time scales, where the forager substantially depletes the patch during its visit and must subsequently decide when to leave the current patch in search of another. A basic result in behavioural ecology, the marginal value theorem (MVT), states that an animal can optimize the resource intake rate by leaving its current patch when the estimated withinpatch resource yield rate falls below the global average resource intake rate of the environment [19]. While this theory has been validated in multiple behavioural studies [20–29], it does not explicitly indicate how beliefs about environmental features (e.g. resource distribution) are accumulated over time or how decision rules might be applied to these beliefs in the form of concrete dynamical equations. To explain how foraging decisions are shaped by the presence and reduction of uncertainty based on resource encounters, it is useful to have normative Bayesian models of patch leaving that ask how animals use limited information to make foraging decisions. However, those that do tend to consider a narrow range of environmental conditions [30,31–46]. General models that have been proposed are challenging to analyse mathematically [47], making it difficult to reveal how environmental parameters shape an optimal forager’s strategy and yield in many different environments.
The aim of our study is thus to develop a Bayesian framework of foraging behaviour, treating decisions as a statistical inference problem and connecting normative theory of foraging decisions with mechanistic evidence accumulation models [48]. Normative strategies provide a bestcase scenario for a particular objective, which can then be used as a touchstone for comparison with heuristics and actual animal behaviours. Animals may actually use evidence accumulation mechanisms that approximate Bayes' optimal strategies, such as energy measurement via the passage of food through the gut [49], so we consider our model to be an abstraction that such physical strategies can be measured against. We first define a general mathematical framework to model patchforaging decisions that applies at different time scales with regards to search and depletion of the resource. These represent the different ecological decision cases described above: ‘habitat choice’ and ‘patch leaving’. Because these cases are only separated in the time scale of resource depletion, we treat both with the framework of patch foraging as an evidence accumulation process whereby a threshold on available evidence triggers a decision.
Using several mathematically tractable cases in which probabilistic updating based on the receipt of resources within a patch can be modelled by stochastic differential equations (SDEs), we determine patchleaving statistics via solutions to firstpassage time problems. We thus obtain analytical expressions for optimal decision thresholds that connect to observable quantities of interest, including patch residence time, travel time, resource consumption over time and patch yield rate over time. For both habitat choice and patch leaving, we show that, in uncertain or resourcepoor environments, uncertainty causes even an ideal Bayesian observer to tend to stay too long in lowyielding patches (overharvesting) and not long enough in highyielding patches (underharvesting). Past studies have suggested that such deviations can also arise from patchdiscrimination limits [18] or behavioural state dependence [50]. Though our work is not the first to demonstrate that over and underharvesting can arise from uncertainty in statistical inference models [51,52], we are able to show how these trends vary across a wide range of environments owing to our model’s mathematical tractability. By establishing a general Bayesian framework for patch foraging at multiple scales, our study provides a platform to study behavioural and neural mechanisms of naturalistic decisionmaking akin to how trained decisionmaking behaviour is studied within systems neuroscience [8,53].
2. Sequential sampling model framework
The patchforaging model framework, which describes both habitat choice and patch leaving, considers an animal searching its environment, which contains distributed resource patches (figure 1a). When the animal enters a patch, it consumes resources within the patch. In the case of habitat choice, we assume that resources are depleted slowly enough that the depletion is negligible, whereas for the patch leaving problem resources are depleted. We represent the decision process to leave a patch via a sequential sampling model for an ideal observer’s posterior of its current patch’s yield rate, λ(t). This assumes that an animal learns over time the yield of the patch it is currently in and to decide if and when it should leave and search for another patch.
The initial yield rate ${\lambda}_{0}^{k}$ determines the rate at which the animal initially encounters a resource in the patch and is drawn from the distribution p(λ_{0}). We assume that the forager knows and initializes its belief with the prior p(λ_{0}) when arriving in a patch (figure 1b). This simplifying assumption allows us to obtain tractable solutions. Considering randomly timed resource encounters within a patch, we use a Poisson rate λ(t) = λ_{0} − ρK(t) generating exponentially distributed waiting times between encounters (as in random search [54]) that decreases with K(t), the number of resource encounters so far, where ρ is the impact of each resource encounter on the underlying yield rate of the patch. Resource encounter history can be described by the summed sequence of encounters, each at time t_{j}: $x(t)={K}^{\prime}(t)=\sum _{\hspace{0.17em}j=1}^{K(t)}\delta (t{t}_{j})$. An ideal forager performs a Bayesian update of its belief about the current patch yield rate λ,
Varying ρ changes the rate of patch depletion relative to the time scale of the foraging process. Small relative values of ρ/λ_{0} represent a large resource patch that the forager depletes very slowly. The limiting case ρ/λ_{0} → 0 represents the habitat choice problem. Alternatively, when this ratio is intermediate up to unity (ρ/λ_{0} ∈ [10^{−2}, 1]), the forager considerably depletes the patch with each encounter. We refer to this as the patch leaving problem, and show that, in such cases, uncertainty in the patch yield can play a major role in shaping the departure strategy. We first consider the habitat choice problem in §3; following this, we consider the patch leaving problem in §4.
3. ‘Habitat choice’: minimizing time to find highresource habitats
Habitat choice refers to patch use at scales where the forager’s activity does not significantly affect the resource distribution or, in other words, that resource depletion occurs very slowly relative to the time needed for the search process. We represent this with the mathematically tractable yet representative limit of zero patch depletion. In this case, the optimal behaviour is to quickly locate a patch with the highest yield and remain there. Although in real environments habitats eventually deplete and the forager would leave, our theoretical treatment of a ‘remain in the highyielding patch’ strategy can simply translate to a ‘stay a long time in the initially highyielding patch’ strategy, with results applying similarly to both because of the separation of time scales: for habitat choice, the time needed to search and decide on a highyield patch to remain in (which we denote as T_{arrive}) is much less than the time that would be needed to actually deplete the patch.
Upon entering a patch, the forager must use its experience of resource encounters to decide whether to stay in the patch or leave for another. We first consider a simplified binary environment where there are only two patch types—highyield versus lowyield—and that the forager knows these possible patch types and their return rates. Here, the optimal behaviour is to infer whether or not it is currently in a highyield patch, and, if so, to stay, but otherwise to leave. Uncertainty and stochasticity of resource encounters means that the forager will visit some lowyielding patches until it learns the yield rate and departs, and may also visit and depart from highyielding patches if the type is incorrectly inferred. We then consider more general cases, and show that the general trends and optimal strategies from the simpler binary case still apply; this includes environments with multiple patch types and environments with continuous distributions of patch types where the forager has a threshold for accepting a patch as sufficiently dense with resources. With this approach, we can explicitly derive statistics associated with patch departures and examine how the efficient identification of highquality habitats depends on environmental parameters like patch discriminability (e.g. λ_{H}/λ_{L}) and highyield patch prevalence (p_{H}).
3.1. Two patch types
An environment with two possible patch types—high yielding and low yielding—is a mathematically tractable case that gives insight into optimal decision strategies and their resulting behavioural observables. Here, the probability distribution of patch types (likelihood that the nextvisited patch is of a certain type) is p_{0}(λ) = p_{H}δ(λ − λ_{H}) + p_{L}δ(λ − λ_{L}): H denotes the higher yielding patch and L denotes the lower yielding patch. As stated, we assume that the forager knows the values λ_{H} and λ_{L} and uses these as prior information to infer the type of the current patch. Using the limit of slow depletion (ρ → 0) to represent habitat choice, the animal determines which patch type it is currently in using the loglikelihood ratio (LLR) y(t) ≡ log(p(λ_{H}x(t))/p(λ_{L}x(t))). With this, their belief update can be written as an SDE:
The longterm resource intake rate is maximized if the forager finds and remains in a highyielding patch (figure 2a). If the forager remains in a highyielding patch, then the energy intake rate will reach λ_{H} in the limit of long time. Before locating and deciding to remain in a highyielding patch, the forager may also visit lowyielding patches, leaving when its belief crosses the threshold (figure 2a), and may also visit and depart from highyielding patches, if they are mistaken for lowyielding ones. The departure threshold sets the certainty that the forager obtains before leaving: a low threshold means high certainty of the patch type before leaving: while a high threshold will result in more departures. Too low a threshold can lead to too much time spent in lowyielding patches while gathering more evidence, while too high a threshold can lead to (incorrectly) departing from highyielding patches before gathering enough evidence to distinguish their type. By setting the optimal threshold that balances uncertainty to minimize the time to arrive and remain in a highyielding patch, we can ask how the environmental characteristics of relative patch yield, relative patch density and travel time influence behaviour and expected return of resources.
The forager’s strategy is determined by the threshold θ on its belief (LLR), given by equation (3.1), shaping the time to find and remain in a highyield patch ${\overline{T}}_{\mathrm{arrive}}(\theta )$. This quantity can be computed from the patchdeparture statistics of the forager, using firstpassage time methods, given the prior y(0) = log(p_{H}/(1 − p_{H})) [57,58]; τ is the mean travel time between patches, which we assume is known or determined from experience. Using this, the time to arrive and remain in a highyield patch is
How should an animal best adapt its habitat search strategy to the statistics of the environment? When highyield patches are rare (low p_{H}), travel times are large (high τ) or patches are easily discriminable (high λ_{H} relative to λ_{L}), the forager should gain higher certainty by deliberating longer before departing a patch; indeed, from equation (3.4) and figure 2d,e we see that the optimal threshold decreases with ρ_{H}, τ and λ_{H}. Increasing discriminability (λ_{H}/λ_{L}) or the highyield patch fraction p_{H} decreases the minimal mean time ${T}_{\mathrm{arrive}}^{\mathrm{opt}}$ needed to arrive and remain in a highyield patch (figure 2f), since this makes finding a highyielding patch easier for the animal.
Furthermore, the outcome ${\overline{T}}_{\mathrm{arrive}}(\theta )$ is most sensitive to the strategy (choice of threshold θ) in environments with low discriminability and a small fraction of desirable patches (figure 2b,c). In experiments, the value of T_{arrive} is an observable that can be used to infer the effective value of θ that an animal is using. The parameter sensitivity suggests that an animal’s patchselection strategy—i.e. the value of θ it is using—could be more precisely inferred when highyield patches are rare or more difficult to identify. Note that the patch leaving rule of thresholding one’s LLR is mathematically equivalent to thresholding the mean estimated resource yield rate since $\overline{\lambda}=({\lambda}_{H}+{\mathrm{e}}^{y}{\lambda}_{L})/(1+{\mathrm{e}}^{y})$, analogous to previous patchdeparture rules developed [31,41]. Next, we generalize this approach to environments with more than two patch types, so decisions use multiple LLRs, such that optimal decisions do not simply map to thresholding the estimated yield rate.
3.2. Multiple patch types
Animals may have to select from any number of patch types in an environment, which begs the question as to how decision and search strategies should extend to more general environments. With multiple patch types, decisions made by computing only two LLRs is sufficient to obtain near optimal performance in terms of minimizing the time to find and remain in a highreturn patch. This result thus complements and extends previous work that has considered optimal strategies for two patch types [33,34] or more general models that are intractable to a mathematical study of how behaviour and yield vary with environmental and strategy parameters [47,59].
To model multiple patch types, consider environments with N patch types having resource yield rates λ_{1} > λ_{2} > · · · > λ_{N} ≥ 0 with patch fractions p_{1}, p_{2}, …, p_{N}. Defining LLRs y_{j} = log(p(λ_{1}x(t))/p(λ_{j+1}x(t))) for j = 1, …, N − 1, yields the N − 1dimensional system fully describing an ideal observer’s belief about the current patch type
As in the binary case, the optimal strategy is to find and remain in the highest yielding patch (λ_{1}). We again represent patch leaving decisions by thresholding the probability of being in the highyielding patch, such that when p(λ_{1}x(t)) = ϕ ∈ (0, p_{1}) the forager exits the patch. We approximate this thresholding process by requiring y_{j} ≥ θ (for j = 1, 2, …, N) to remain in the patch, so the forager departs given sufficient evidence that it is not in the highest yielding patch (see figure 3a for three patch types).
We also consider how effective reduced strategies are, for which the observer only tracks the first L LLRs y_{1}, y_{2}, …, y_{L} and compares these with the threshold θ to decide when to leave the patch. Thus, we compute the mean time to arrive and remain in the highyield patch, which depends on the escape probability π_{1}(θ) from the highyielding patch and the mean time to visit each patch ${\overline{T}}_{j}(\theta ,L)$ when escaping,
The mean highyield patch arrival time ${\overline{T}}_{\mathrm{arrive}}$ depends strongly on the highyield patch resource yield rate λ_{1}, which decreases considerably as the patch becomes more discriminable (three patches: figure 3b; five patches: figure 3d). On the other hand, ${\overline{T}}_{\mathrm{arrive}}$ depends weakly on the worst patch’s yield rate λ_{3} (figure 4a), so uncertainty among the less valuable patches has little effect on behaviour. In a related way, ${\overline{T}}_{\mathrm{arrive}}$ is much more strongly affected by changes in the fraction of the highyielding patch (p_{1}: figure 3c) than by changes in the balance of the mid (λ_{2}) and lowyielding (λ_{3}) patches (figure 4b).
Again, the optimal threshold decreases when patches are more discriminable: as λ_{1} increases the forager should gain a higher certainty before leaving (figure 3b). The average highyield patch arrival time ${\overline{T}}_{\mathrm{arrive}}$ depends weakly on the threshold near the optimum, but the optimum threshold shows a nonmonotonic dependence on p_{1}. The lower optimum threshold for both low and highyield p_{1} values represents that, in these cases, it is optimal to be more certain of the future harvest rate before leaving: for low p_{1} this occurs because highyield patches are rare (and thus there is a higher premium on distinguishing the highyielding patch when actually in one), and for high p_{1} this occurs because they are plentiful (one is more likely to land in a highyielding patch, so one can afford to require more certainty to depart). Such an increased premium placed on information gathering for the reduction of uncertainty about the future yield of a patch in sparse environments has been identified in previous analyses of foraging models governed by statistical decision theory [59,60]. Between these cases, although the optimal threshold is slightly higher, the dependence is weak. Additionally, this demonstrates that, if the forager did not know p_{1} (we assumed that this is known and is used to formulate the leaving decisions), the best strategy would be to err on the side of choosing a low threshold, because the sensitivity of ${\overline{T}}_{\mathrm{arrive}}$ to threshold is relatively weak for choices too low but can be higher for choices too high (figure 3c).
In an environment with five patches, performance depends weakly on how many LLRs are used to make patchleaving decisions for L > 2. It is sufficient to simply track the LLRs between the first three patches, which correspond to using L = 2 (figure 3d). This is because the forager only needs to know whether it is in one of the best patches or not, since the goal is to eventually settle in one such patch as a habitat. This demonstrates again that the key features of uncertainty that matter to the optimal forager are the discriminability and prevalence of the best and second best patch type.
3.3. Continuum limit: many patch types
Building on the Npatch case, we now consider a scenario where there is a continuous distribution of patch qualities (N → ∞), so the resource yield rate for each patch λ is drawn from a continuous distribution p_{0}(λ), which serves as a prior for the posterior p(λx(t)) with each patch visit (figure 5a). For any continuous probability distribution function p_{0}(λ), the maximum λ will never be sampled, so arriving and remaining in the ‘maximum’ yielding patch is not possible. We therefore assume that the forager seeks patches with yield rates λ_{θ} or above, but deems lower yield rates to be insufficient. With this formulation, the forager updates an LLR based on a belief of whether the current patch is greater or less than λ_{θ}. Because this divides the continuous distribution into two categories, the mathematical treatment is similar to the binary case, but with added uncertainty because the patches in each category do not have the same return.
To model this, given a reference yield rate λ_{θ}, we represent decisions in an environment with a continuous distribution of patch qualities by tracking $P(\lambda >{\lambda}_{\theta}x(t))={\int}_{{\lambda}_{\theta}}^{\mathrm{\infty}}p(\lambda x(t))\hspace{0.17em}\mathrm{d}\lambda $. For the case of an exponential prior p_{0}(λ) = αe^{−αλ}, given K(t) resource encounters, we define ρ(t) = log (P(λ > λ_{θ}x(t))/P(λ < λ_{θ}x(t))) and state that the forager departs the patch when $\rho (t)\le \hat{\phantom{\rule{2.8pt}{0ex}}\theta}$ or when $P(\lambda >{\lambda}_{\theta}x(t))\le \theta \hspace{0.17em}:\hspace{0.17em}=1/(1+{\mathrm{e}}^{\hat{\phantom{\rule{2.8pt}{0ex}}\theta}})$. Note that, to allow evidence accumulation, we require that $\theta <\alpha {\int}_{{\lambda}_{\theta}}^{\mathrm{\infty}}{\mathrm{e}}^{\alpha \lambda}\hspace{0.17em}\mathrm{d}\lambda ={\mathrm{e}}^{\alpha {\lambda}_{\theta}}\equiv \varphi $, which represents the fraction of patches where λ ≥ λ_{θ}.
Computing the probability of escaping a highyield patch, ${\pi}_{H}(\theta ;{\lambda}_{\theta})={\int}_{{\lambda}_{\theta}}^{\mathrm{\infty}}{p}_{0}(\lambda )\pi (\theta ;\lambda )\hspace{0.17em}\mathrm{d}\lambda $, and the mean time per visit to high and lowyield patch types, ${\overline{T}}_{H}(\theta ;{\lambda}_{\theta})={\int}_{{\lambda}_{\theta}}^{\mathrm{\infty}}{p}_{0}(\lambda )\overline{T}(\theta ;\lambda )\hspace{0.17em}\mathrm{d}\lambda $ and ${\overline{T}}_{L}(\theta ;{\lambda}_{\theta})={\int}_{0}^{{\lambda}_{\theta}}{p}_{0}(\lambda )\overline{T}(\theta ;\lambda )d\lambda $, when departing (using Monte Carlo sampling), we then use equation (3.2) to compute the time to arrive in a highyielding patch (figure 5b). Placing a higher threshold λ_{θ} on the quality of an acceptably highyielding patch increases the time to arrive in an acceptably highyield patch. Moreover, the optimal threshold θ decreases, as more time must be spent in patches to discriminate a highyielding patch, which become rarer as λ_{θ} increases. Increasing λ_{θ} corresponds to making sufficiently highyield patches more discriminable and more rare.
With this formulation, the mathematical treatment in the case of a continuous distribution of patches is then the same as the binary case, and we can map corresponding results: setting a higher ${\lambda}_{\theta}$ is equivalent to decreasing p_{H} and concurrently increasing λ_{H}. Although we considered an exponential distribution for p_{0}(λ), we note that if this distribution changes, then this will affect the relationship between ${\lambda}_{\theta}$ and the equivalent mapping onto the binary case (in terms of p_{H} and λ_{H}). Another possibility would be to further ‘bin’ the continuous distribution to correspond to three effective types, instead of two, as we did using a single threshold. The continuous case then would be treated analogously to a threepatchtype environment, and could be represented with two LLRs. However, further binning may not be necessary to achieve nearoptimal decisions. Overall, this demonstrates that effective strategies for foraging environments with a continuum of patch types could be generated using particle filters that compute likelihoods over a finite set of patch types [61].
3.4. Summary of results: habitat choice problem
In general, we see that strategies that divide the environment into two patch types work well in efficiently finding the best or nearbest patches, even in the presence of many patch types. The optimal time to arrive and remain in the highest yielding patch decreases as the highyield patch discriminability increases and as highyield patches become more common. Considering more than two patch types, the associated foraging strategies are most strongly coupled to environmental parameters of the highest and second highest yielding patch types. It is not necessary to compute LLRs associated with all possible types in order to efficiently find a highyield patch—even considering only a single LLR gives reasonable results, and the average time to arrive in a highyield patch is not strongly affected when the number of LLRs continues to increase beyond two. This suggests that animals select habitats by estimating a possible range of highquality patches and then making patchdeparture decisions based on whether patches meet those criteria or not.
4. Patch leaving: depletion versus uncertaintydriven decisions
When the scale of a patch is smaller, the forager will significantly deplete the patch’s resources during its visit. The decision is then not of which patch to remain in, but rather of when to leave the current patch in search of another. We therefore refer to this as patch exploitation (figure 1a; [12,62]). While the nature of the resource differs for different animals (and typical patch residence times can accordingly vary from seconds up to hours—for some examples, see [30,36,45,63,64]). These cases all have in common that each resource patch is small enough that availability within the patch is affected by the consumption of the forager.
The MVT sets the optimal time to leave a patch in order to maximize resource consumption over time: when the current patch yield rate equals the overall average yield rate for the environment [19]. However, the MVT is simply an optimal rule, and does not specify the mechanistic process of how an animal uses its experience to reach a decision to leave a patch. Previous work has demonstrated that rewards in discrete chunks—instead of as a continuous rate—can affect the process an animal uses in decisionmaking [21,24,48]. From a Bayesian perspective, decisions should use available information about the resource distribution in the environment. If resource availability within a patch is discrete or uncertain, even an ideal observer may not be able to accurately infer the actual rate of return, and thus would not be able to implement the leaving rule prescribed by the MVT. Experiments show that, while the general trends predicted by the MVT hold in many cases, animals often deviate from an MVTpredicted strategy [50]. Moreover, in cases where patches contain very few items (e.g. 0 or 1 resource chunks), reward is not described by a rate function, and the MVT leaving rule does not apply.
Here we consider an animal that encounters resources in discrete chunks and infers the state of the environment and subsequently acts. This allows us to ask when the MVT rule is actually optimal versus when it does not apply, when deviations from the MVT occur owing to uncertainty and how a forager can incorporate prior knowledge about the resource distribution in the environment to reach a patch leaving decision. We first treat the simple case of homogeneous patch types to establish the basic theoretical approach. Then, we consider an environment with two patch types to show how the inference procedure affects decisions in different environmental configurations, which we refer to as the ‘depletiondominated’ versus ‘uncertaintydominated’ regimes.
4.1. Homogeneous environments
To show how discreteness of resources affects decisions [21,24], we first consider the simple case of a homogeneous environment with a single patch type. An ideal forager with prior knowledge of the initial yield rate λ_{0} can track time and resource encounters to determine the current yield rate λ(t), and then depart the patch when the inferred value of λ(t) falls below some threshold ${\lambda}_{\theta}$. Prior knowledge of the initial patch yield can be used in order to infer
Using this inference strategy, we calculate the longterm resource intake rate by assuming that λ_{θ} is an integer multiple of ρ. With this, the number of chunks consumed before departure is m_{θ} ≡ K(T(λ_{θ})) : = (λ_{0} − λ_{θ})/ρ. Linearity of expectations allows us to compute the mean departure time as the sum of mean exponential waiting times between resource encounters ${T}_{{\lambda}_{\theta}}=[{H}_{{m}_{0}}{H}_{{m}_{0}{m}_{\theta}}]/\rho $, where H_{n} is the nth harmonic number. Thus, we can approximate the longterm resource intake rate given λ_{θ} as
4.2. Binary environments
In binary environments, the forager estimates the underlying yield rate of the current patch, $\stackrel{~}{\lambda}(t)$, and departs when this falls below a threshold λ_{θ}. Constant threshold strategies are our focus owing to their relative simplicity, but we note that alternatively dynamic programming could be used to determine optima of a more general class of departure strategies [65]. We assume that the forager uses knowledge that two different patch types exist to estimate the yield rate of the current patch; this involves using prior information to discriminate the patch type (high or low), combined with resource encounters which decrement the estimated yield rate. The belief can be determined according to a nonautonomous SDE for an LLR,
We focus on two different scenarios, which we refer to as: (i) depletiondominated regime, where the initial yield rate of the patch is known, and therefore leaving decisions are based solely on depletion, and (ii) uncertaintydominated regime, where the type of patch is not known upon entry, and optimal leaving decisions must consider uncertainty in the estimate of the current yield rate of the patch. For the uncertaintydominated regime, we consider first the cases where lowreturn patches contain zero resources, and then generalize to different amounts of resources per patch type.
Depletiondominated regime. To represent what we term the depletiondominated regime, we assume that the forager arrives in a patch and immediately knows the patch type λ_{j} (j ∈ {H, L}) in which it resides (e.g. owing to information provided by conspecifics or visual cues). In this case, the forager can make an accurate estimate of the true underlying yield rate of the patch, and therefore there is no uncertainty. Thus, leaving decisions are driven by depletion of the patch, as determined by when the estimated yield rate falls below some level ${\lambda}_{\theta}^{j}$.
Following our calculations from the homogeneous case, the longterm resource intake rate depends on the initial resource chunk count in patches of type j, ${m}_{0}^{j}$, and the departure thresholds, so in the large ${m}_{0}^{j}$ limit
Uncertaintydominated regime—empty lowyield patch. In the ‘uncertaintydominated’ regime, the forager does not know the initial yield rate of the patch upon entry. However, we assume that it has prior knowledge of the types of patches in the environment, i.e. that it knows the values of λ_{H} and λ_{L}. We first consider the tractable scenario where the lowyielding patch is empty (λ_{L} = 0). Such situations occur if certain regions of the environment appear to have food (e.g. fruiting vegetation) but on closer inspection turn out to be empty (e.g. already foraged or rotten). The optimal strategy is for the observer to first wait a finite time T_{θ} to depart if no resources are encountered, but if resources are encountered before t = T_{θ} to consume those resources until the inferred yield rate drops to λ_{θ} = (m_{0} − m_{θ})ρ. Early/late decisions are thus driven by uncertainty/depletion. Assuming m_{0} ≫ 1, we can continuously approximate the longterm resource intake rate and find that it is maximized using a waiting time T_{θ} that is insensitive to p_{H}. However, the threshold λ_{θ} depends on both p_{H} and travel time, because these parameters affect the overall average rate of resources available in the environment (figure 6c). The optimal threshold ${\lambda}_{\theta}^{\mathrm{opt}}$ is not specified by the MVT, since uncertainty drives the forager to spend nonzero time in empty patches, adding extraneous time to the foraging process.
Many resource chunks. Next, we generalize to examine binary environments in which ${m}_{0}^{H}>{m}_{0}^{L}$ are arbitrary integers. In this case, the belief y(t) = log (P(λ_{H} − K(t)ρx(t))/P(λ_{L} − K(t)ρx(t))) evolves according to equation (4.4). The forager estimates the current yield rate of the patch from this belief,
Comparing cases, we see that foragers in sparser environments (lower average initial resource amount ${\overline{m}}_{0}=({\overline{m}}_{0}^{H}+{\overline{m}}_{0}^{L})/2$) stay in lowyield patches too long and leave highyield patches too soon in comparison with observers that immediately know their patch type, owing to their uncertainty about their current patch before and at the time of departure (figure 7c). Uncertainty thus drives animals to underexploit (overexploit) high (low)yielding patches when high and lowyield patches are different enough that it is optimal to spend significantly more time in high versus lowyield patches, but similar enough as to not be immediately distinguishable. Optimal leaving decisions in the uncertaintydominated regime must use a rateestimation process, because of the associated uncertainty in the true yield rate of the current patch.
5. Discussion
Patch foraging is a rich and flexible behaviour where an animal enters a patch of resources, harvests them and then leaves to search for another patch. An animal’s behaviour can be quantified by its patch residence time distribution, travel time distribution, the amount of resources consumed and the movement pattern between patches. In this work, we used principles of probabilistic inference to establish a normative theory of patchleaving decisions. With this general framework, we showed how foraging at different temporal and spatial scales is connected by a similar decision problem: ‘habitat choice’ refers to larger scales when foragers do not significantly deplete a resource, and ‘patch exploitation’ refers to smaller scales when the forager’s activity depletes the patch. For habitat choice the optimal behaviour is to quickly locate and remain in a highyielding habitat, while for patch exploitation it is optimal to use prior information along with reward encounters to estimate the current underlying yield rate to determine when to leave the patch (figure 8 and table 1).
environment  decision strategy and dependencies  equations  figures 

habitat selection  
objective: minimize time to find highest yielding habitat  
known: resource yield rates of each habitat type  
Nhabitat types 

N = 2: equation (3.1); N ≥ 3: equation (3.5)  N = 2: figure 2; N ≥ 3: figure 3 
continuum of habitat types 

figure 5  
patch exploitation  
objective: maximize mean resource intake rate R over a long time (several patches)  
known: initial yield rates of each patch type  
1patch type 

equation (4.3)  figure 6a 
2patch types: patch type known on arrival 

equations (4.5) and (4.6)  figure 6b 
2patch types: empty lowyield patch 

figure 6c  
2patch types: both high and lowyield patches have resources 

equation (4.7)  figure 7 
In ecological contexts, these activities are part of a behavioural hierarchy, where an animal must decide where to forage and how long to exploit a certain resource.
In the case of habitat choice, the forager should use its experience of reward encounters to determine whether to stay or leave; in our model an optimal forager departs a habitat when its LLR for the probability of highyield versus other habitats falls below a threshold. Optimal decisions are based on inference of habitat quality, with uncertainty being the driving factor in habitat departure times; while this is related to resource intake rates, it is not the same, because of how prior information can be used in patch inference. We showed that, with multiple different patch types, it is not necessary to track LLRs for all patch types—behaviour is most strongly affected by inference related to the highest and secondhighest yield habitat types. The optimal time to arrive and remain in a highyielding habitat is lower when patches are more discriminable, or when highyield patches are more common. While T_{arrive} is an experimentally observable quantity, an animal’s internal decision threshold is not; our model connects these quantities, and thus can be used to infer the decision rules an animal is using (figure 2). For example, a similar approach has been very informative to infer the parameters underlying twochoice decision tasks [66]. We showed that it is optimal to have a lower threshold—and thus gain a higher certainty before leaving—when travel times are large, highyield patches are rare or highyield patches are easier to discriminate. Analogous results have been found in observations of animals seeking habitats, where dispersal costs affect whether or not animals search for a new habitat [67]. These results give quantitative predictions that can be used to interpret experiments, for example to examine whether the animal is minimizing the time to reach the highest yield habitat in a heterogeneous environment. Moreover, we showed how behaviour in the general case where many habitat types exist can be understood by mapping results onto the tractable case of only two different patch types (figures 3 and 5). By varying systematically the percentage of highyielding habitats and the discriminability (ratio of high to lowyield rate), the model predicts how this affects the minimal time to arrive at the highest yield habitat, and connects this to a process that could implement such computations.
For patch exploitation, when the forager depletes patches in its habitat, in most cases the longterm intake rate is maximized by departing a patch when the inpatch estimated resource yield rate matches the average return rate of the environment (i.e. by implementing the MVT rule). However, this does not apply when resources within a patch are limited so there is more uncertainty about the yield of the current patch upon departure (figure 7). Often in Nature, the environment is volatile and animals make foraging decisions while uncertain about local resource availability [68]. Our model predicts that, if there is high uncertainty about the patch type, this causes even an ideal Bayesian forager to stay longer in lowyield patches and shorter in highyield patches than predicted by the MVT.
Our theoretical treatment of patchleaving decisions builds on previous Bayesian models of foraging [31–39,41]. Our approach goes further than previous work by providing a stepbystep derivation of the normative strategies associated with a continuum of different environmental conditions, systematically identifying the dependence of observable behaviours (e.g. patchdeparture times) on environmental parameters. During habitat choice, the minimal arrival time to the highyield habitat scales with the probability of highyield patches in the environment. On the other hand, we have shown that, in the case of depleting patches, the amount of time a forager overstays or understays in a patch scales with the density of resources in the patch. We are also able to infer the optimal threshold an animal should use. While this cannot be measured directly in experiments, our observations do reveal environmental parameter regimes under which performance (e.g. foraging yield) is sensitive to changes in strategy. This not only informs the design of behavioural foraging experiments, so as to determine task parameters that best reveal an animal’s strategy, but optimal yields can also be compared with those obtained by animals in the wild to see how finely tuned their foraging strategies are.
Analysis of experiments shows that animals forage in ways that suggest they use Bayesian reasoning [40–46], using prior knowledge of their environment to modulate foraging behaviour [64,69,70]. For example, bumblebees [30] and Inca doves [71] adjust their foraging strategies in response to the predictability of the environment, as a Bayesian forager would, but this is not a universal trend [72]. Patchleaving decisions may deviate from Bayes' optimality as animals become riskaverse in variable environments [73]. Other Bayesian models have considered patchforaging decisions, even in rich multiple patch environments [34,47], but this work does not necessarily systematically vary environmental and strategy parameters to explore how the sensitivity of performance changes. Our work is also sufficiently mathematically tractable to suggest a mechanistic implementation that the forager can use to implement optimal decision rules. Moreover, our theoretical approach applies not only to patch exploitation, but also to habitat choice—where the MVT does not apply—and thus enables connections across these multiple scales of behaviour [13].
Although we used a constant threshold value based on either the belief or estimated yield rate, other work has examined cases where optimal decision strategies involve timedependent decision thresholds [65,74–76]. Typically, these results arise in the context of multitrial experiments in which the quality of evidence on each trial varies stochastically and is initially unknown. In the habitatchoice context, the quality of evidence is fixed across habitat visits, fitting the assumptions of classic, constant threshold optimal policies. We would therefore expect an analysis allowing for a dynamic threshold to yield the same results as we obtained here. On the other hand, when the animal performed patch exploitation in uncertain binary environments, we projected a higher dimensional description of the patch value to a single scalar estimate of the patch yield rate. In this case, a constantthreshold implementation may not be purely optimal. Leveraging methods from dynamic programming commonly used to set optimal decision policies [75] would be a fruitful next step in ensuring the optimality of our patchleaving decision strategies. A common theme in previous Bayesian models that use dynamic programming [41,47] and our approach is that the forager should use the expected future return, not the current return, to make departure decisions. However, note that current and future return should be tightly monotonically related, especially in the limit of short times into the future. An advantage to the constantthreshold treatment is that it enables simple explicit quantitative relations that can be used to interpret experimental data (figure 2). For example, experiments evaluating how animals value the quality of evidence could help us delineate whether the animal is using a constant threshold or not.
Effective search is integral to survival in Nature [77], and search behaviour can give information on individual decision strategies. One can consider search within and between patches. Although we assumed random timings of resource encounters, an extension of our model could take into account different spatial arrangements. Within patches, an animal may perform random or systematic searches. For example, a recent study found that rats can solve the stochastic travelling salesman problem using a nearest neighbour algorithm [78]. A similar approach could ask how an animal’s search and navigation pattern interacts with different patchleaving decisions to create an effective foraging strategy that also considers memory of specific patch locations. Indeed, the explicit consideration of spatial movement may be necessary to understand foraging decisions. Previous work found that, when rats must physically move to perform foraging, the observed behaviour differed from tasks that ‘simulate’ foraging by presenting sequential choices or that consider a visual search [79]. It is an open question as to how the animal integrates aspects of spatial movement with economic valuations of future reward.
Our model assumes that animals know the initial yield rate of each type of patch in the environment. In a reallife context or in an experimental setup, the animal would learn the environmental parameters, which we could model by considering another level in the inference hierarchy whereby the patchtype quality and fraction are learned along with the transit time distribution. Although we considered a single forager acting alone, another important extension will be to consider interactions between animals, either through predator–prey interactions which affect foraging decisions [80], social foraging of groups [52,81] or even competitive foraging [17,18,82]. In our model, the forager only receives direct (nonsocial) information about resource availability; in collective foraging, an individual receives both social and nonsocial information [83,84]. This can significantly affect foraging decisions, for example in the case where an individual must balance resourceseeking with group cohesion. Building on our modelling approach, foragers could share social information either by cooperating in the inference of the patch quality or by signalling to each other when to depart a patch as a threshold is reached.
To conclude, our model establishes a formal framework for the quantitative analysis of a natural behaviour—patch foraging (involving both habitat choice and patch exploitation)—that can be studied with the same formal rigor as many trained behavioural tasks. Such validated behavioural algorithms are crucial for the systematic design of future experiments and interpretation of data on animal behaviour [85]. By comparing with theoretical optimal strategies, experiments and data can be used to understand the decision strategies an animal is employing and relate these to recorded animal movement and neural data. Future work will build on this model framework to generate testable hypotheses on the role of social interactions and the neural mechanistic underpinnings of foraging behaviour.
Data accessibility
No data were collected for or used in this paper.
Authors' contributions
Z.P.K., J.D.D. and A.E.H. conceived the project; Z.P.K. performed the model simulations and analysis and drew the figures; J.D.D. and A.E.H. performed a literature review of past foraging models and experiments; all authors wrote the manuscript.
Competing interests
We declare we have no competing interests.
Funding
Z.P.K. was supported by a CRCNS grant no. (R01MH115557) and NSF (DMS1853630). A.E.H. was supported by NIH grant no. 1R21MH12188901. J.D.D. was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy—EXC 2117—422037984 and the Heidelberg Academy of Science.
Acknowledgements
We thank Jan Drugowitsch for helpful discussions regarding the optimality of decision threshold policies across patch visits.
References
 1.
Calhoun AJ, Chalasani SH, Sharpee TO . 2014 Maximally informative foraging by Caenorhabditis elegans. eLife 3, e04220. (doi:10.7554/eLife.04220) Crossref, Web of Science, Google Scholar  2.
Landayan D, Feldman DS, Wolf FW . 2018 Satiation statedependent dopaminergic control of foraging in Drosophila. Sci. Rep. 8, 19. (doi:10.1038/s41598018242171) Crossref, PubMed, Web of Science, Google Scholar  3.
Marsh B, SchuckPaim C, Kacelnik A . 2004 Energetic state during learning affects foraging choices in starlings. Behav. Ecol. 15, 396399. (doi:10.1093/beheco/arh034) Crossref, Web of Science, Google Scholar  4.
Carter EC, Redish AD . 2016 Rats value time differently on equivalent foraging and delaydiscounting tasks. J. Exp. Psychol.: General 145, 1093. (doi:10.1037/xge0000196) Crossref, PubMed, Web of Science, Google Scholar  5.
Rose LM . 1994 Sex differences in diet and foraging behavior in whitefaced capuchins (Cebus capucinus). Int. J. Primatol. 15, 95114. (doi:10.1007/BF02735236) Crossref, Web of Science, Google Scholar  6.
Hills TT . 2006 Animal foraging and the evolution of goaldirected cognition. Cogn. Sci. 30, 341. (doi:10.1207/s15516709cog0000_50) Crossref, PubMed, Web of Science, Google Scholar  7.
Raine NE, Ings TC, Dornhaus A, Saleh N, Chittka L . 2006 Adaptation, genetic drift, pleiotropy, and history in the evolution of bee foraging behavior. Adv. Study Behav. 36, 305354. (doi:10.1016/s00653454(06)36007x) Crossref, Web of Science, Google Scholar  8.
Mobbs D, Trimmer PC, Blumstein DT, Dayan P . 2018 Foraging for foundations in decision neuroscience: insights from ethology. Nat. Rev. Neurosci. 19, 419. (doi:10.1038/s4158301800107) Crossref, PubMed, Web of Science, Google Scholar  9.
Hayden BY, Walton ME . 2014 Neuroscience of foraging. Front. Neurosci. 8, 81. (doi:10.3389/fnins.2014.00081) Crossref, PubMed, Web of Science, Google Scholar  10.
HallMcMaster S, Luyckx F . 2019 Revisiting foraging approaches in neuroscience. Cogn. Affect. Behav. Neurosci. 19, 225230. (doi:10.3758/s1341501800682z) Crossref, PubMed, Web of Science, Google Scholar  11.
Calhoun AJ, Hayden BY . 2015 The foraging brain. Curr. Opin. Behav. Sci. 5, 2431. (doi:10.1016/j.cobeha.2015.07.003) Crossref, Web of Science, Google Scholar  12.
Stephens DW . 2008 Decision ecology: foraging and the ecology of animal decision making. Cogn. Affect. Behav. Neurosci. 8, 475484. (doi:10.3758/CABN.8.4.475) Crossref, PubMed, Web of Science, Google Scholar  13.
McGarigal K, Wan HY, Zeller KA, Timm BC, Cushman SA . 2016 Multiscale habitat selection modeling: a review and outlook. Landscape Ecol. 31, 11611175. (doi:10.1007/s109800160374x) Crossref, Web of Science, Google Scholar  14.
Meyer CB, Thuiller W . 2006 Accuracy of resource selection functions across spatial scales. Divers. Distrib. 12, 288297. (doi:10.1111/j.13669516.2006.00241.x) Crossref, Web of Science, Google Scholar  15.
Kacelnik A, Vasconcelos M, Monteiro T, Aw J . 2011 Darwin’s ‘tugofwar’ vs. starlings’ ‘horseracing’: how adaptations for sequential encounters drive simultaneous choice. Behav. Ecol. Sociobiol. 65, 547558. (doi:10.1007/s0026501011012) Crossref, Web of Science, Google Scholar  16.
Boyce MS, Vernier PR, Nielsen SE, Schmiegelow FK . 2002 Evaluating resource selection functions. Ecol. Modell. 157, 281300. (doi:10.1016/S03043800(02)002004) Crossref, Web of Science, Google Scholar  17.
Kennedy M, Gray RD . 1993 Can ecological theory predict the distribution of foraging animals? A critical analysis of experiments on the ideal free distribution. Oikos 68, 158166. (doi:10.2307/3545322) Crossref, Web of Science, Google Scholar  18.
Abrahams MV . 1986 Patch choice under perceptual constraints: a cause for departures from an ideal free distribution. Behav. Ecol. Sociobiol. 19, 409415. (doi:10.1007/BF00300543) Crossref, Web of Science, Google Scholar  19.
Charnov EL . 1976 Optimal foraging, the marginal value theorem. Theor. Popul. Biol. 9, 129136. (doi:10.1016/00405809(76)90040X) Crossref, PubMed, Web of Science, Google Scholar  20.
Waage JK . 1979 Foraging for patchilydistributed hosts by the parasitoid, Nemeritis canescens. J. Anim. Ecol. 48, 353371. (doi:10.2307/4166) Crossref, Web of Science, Google Scholar  21.
McNamara J . 1982 Optimal patch use in a stochastic environment. Theor. Popul. Biol. 21, 269288. (doi:10.1016/00405809(82)900181) Crossref, Web of Science, Google Scholar  22.
Mellgren RL . 1982 Foraging in a simulated natural environment: there’s a rat loose in the lab. J. Exp. Anal. Behav. 38, 93100. (doi:10.1901/jeab.1982.3893) Crossref, PubMed, Web of Science, Google Scholar  23.
Cuthill I, Kacelnik A . 1990 Central place foraging: a reappraisal of the ‘loading effect’. Anim. Behav. 40, 10871101. (doi:10.1016/S00033472(05)801757) Crossref, Web of Science, Google Scholar  24.
Rita H, Ranta E . 1998 Stochastic patch exploitation model. Proc. R. Soc. Lond. B 265, 309315. (doi:10.1098/rspb.1998.0297) Link, Web of Science, Google Scholar  25.
Driessen G, Bernstein C . 1999 Patch departure mechanisms and optimal host exploitation in an insect parasitoid. J. Anim. Ecol. 68, 445459. (doi:10.1046/j.13652656.1999.00296.x) Crossref, Web of Science, Google Scholar  26.
Ward JF, Austin RM, Macdonald DW . 2000 A simulation model of foraging behaviour and the effect of predation risk. J. Anim. Ecol. 69, 1630. (doi:10.1046/j.13652656.2000.00371.x) Crossref, Web of Science, Google Scholar  27.
Eliassen S, Jørgensen C, Mangel M, Giske J . 2009 Quantifying the adaptive value of learning in foraging behavior. Am. Nat. 174, 478489. (doi:10.1086/605370) Crossref, PubMed, Web of Science, Google Scholar  28.
Taneyhill DE . 2010 Patch departure behavior of bumble bees: rules and mechanisms. Psyche 2010, 872736. (doi:10.1155/2010/872736) Google Scholar  29.
Zhang F, Hui C . 2014 Recent experiencedriven behaviour optimizes foraging. Anim. Behav. 88, 1319. (doi:10.1016/j.anbehav.2013.11.002) Crossref, Web of Science, Google Scholar  30.
Biernaskie JM, Walker SC, Gegear RJ . 2009 Bumblebees learn to forage like Bayesians. Am. Nat. 174, 413423. (doi:10.1086/603629) Crossref, PubMed, Web of Science, Google Scholar  31.
McNamara JM, Green RF, Olsson O . 2006 Bayes’ theorem and its applications in animal behaviour. Oikos 112, 243251. (doi:10.1111/j.00301299.2006.14228.x) Crossref, Web of Science, Google Scholar  32.
Olsson O, Holmgren NMA . 1998 The survivalratemaximizing policy for Bayesian foragers: wait for good news. Behav. Ecol. 9, 345353. (doi:10.1093/beheco/9.4.345) Crossref, Web of Science, Google Scholar  33.
McNamara J, Houston A . 1980 The application of statistical decision theory to animal behaviour. J. Theor. Biol. 85, 673690. (doi:10.1016/00225193(80)902659) Crossref, PubMed, Web of Science, Google Scholar  34.
Olsson O . 2006 Bayesian foraging with only two patch types. Oikos 112, 285297. (doi:10.1111/j.00301299.2006.13549.x) Crossref, Web of Science, Google Scholar  35.
Pierre JS, Green RF . 2008 A Bayesian approach to optimal foraging in parasitoids. Behavioral ecology of insect parasitoids: from theoretical approaches to field applications, pp. 357–383. Oxford, UK: Blackwell. Google Scholar  36.
Olsson O, Holmgren NMA . 1999 Gaining ecological information about Bayesian foragers through their behaviour. I. Models with predictions. Oikos 81, 251263. (doi:10.2307/3546740) Crossref, Web of Science, Google Scholar  37.
Van Gils JA . 2010 Statedependent Bayesian foraging on spatially autocorrelated food distributions. Oikos 119, 237244. (doi:10.1111/j.16000706.2009.17497.x) Crossref, Web of Science, Google Scholar  38.
RodriguezGirones MA, Vasquez RA . 1997 Densitydependent patch exploitation and acquisition of environmental information. Theor. Popul. Biol. 52, 3242. (doi:10.1006/tpbi.1997.1317) Crossref, PubMed, Web of Science, Google Scholar  39.
Ellison AM . 2004 Bayesian inference in ecology. Ecol. Lett. 7, 509520. (doi:10.1111/j.14610248.2004.00603.x) Crossref, Web of Science, Google Scholar  40.
Alonso JC, Alonso JA, Bautista LM, MuñozPulido R . 1995 Patch use in cranes: a field test of optimal foraging predictions. Anim. Behav. 49, 13671379. (doi:10.1006/anbe.1995.0167) Crossref, Web of Science, Google Scholar  41.
Green RF . 1980 Bayesian birds: a simple example of Oaten’s stochastic model of optimal foraging. Theor. Popul. Biol. 18, 244256. (doi:10.1016/00405809(80)900519) Crossref, Web of Science, Google Scholar  42.
Olsson O, Wiktander U, Holmgren NM, Nilsson SG . 1999 Gaining ecological information about Bayesian foragers through their behaviour. II. A field test with woodpeckers. Oikos 87, 264276. (doi:10.2307/3546741) Crossref, Web of Science, Google Scholar  43.
Valone TJ . 2006 Are animals capable of Bayesian updating? An empirical review. Oikos 112, 252259. (doi:10.1111/j.00301299.2006.13465.x) Crossref, Web of Science, Google Scholar  44.
Killeen PR, Palombo GM, Gottlob LR, Beam J . 1996 Bayesian analysis of foraging by pigeons (Columba livia). J. Exp. Psychol.: Animal Behav. Process. 22, 480. (doi:10.1037/00977403.22.4.480) Crossref, PubMed, Google Scholar  45.
Nonacs P, Soriano JL . 1998 Patch sampling behaviour and future foraging expectations in Argentine ants, Linepithema humile. Anim. Behav. 55, 519527. (doi:10.1006/anbe.1997.0615) Crossref, PubMed, Web of Science, Google Scholar  46.
Valone TJ, Brown JS . 1989 Measuring patch assessment abilities of desert granivores. Ecology 70, 18001810. (doi:10.2307/1938113) Crossref, Web of Science, Google Scholar  47.
Green RF . 2006 A simpler, more general method of finding the optimal foraging strategy for Bayesian birds. Oikos 112, 274284. (doi:10.1111/j.00301299.2006.13462.x) Crossref, Web of Science, Google Scholar  48.
Davidson JD, El Hady A . 2019 Foraging as an evidence accumulation process. PLoS Comput. Biol. 15, e1007060. (doi:10.1371/journal.pcbi.1007060) Crossref, PubMed, Web of Science, Google Scholar  49.
Higginson AD, Fawcett TW, Houston AI, McNamara JM . 2018 Trust your gut: using physiological states as a source of information is almost as effective as optimal Bayesian learning. Proc. R. Soc. B 285, 20172411. (doi:10.1098/rspb.2017.2411) Link, Web of Science, Google Scholar  50.
Nonacs P . 2001 State dependent behavior and the marginal value theorem. Behav. Ecol. 12, 7183. (doi:10.1093/oxfordjournals.beheco.a000381) Crossref, Web of Science, Google Scholar  51.
Oaten A . 1977 Optimal foraging in patches: a case for stochasticity. Theor. Popul. Biol. 12, 263285. (doi:10.1016/00405809(77)900466) Crossref, PubMed, Web of Science, Google Scholar  52.
Clark CW, Mangel M . 1984 Foraging and flocking strategies: information in an uncertain environment. Am. Nat. 123, 626641. (doi:10.1086/284228) Crossref, Web of Science, Google Scholar  53.
Gabay AS, Apps MAJ . 2020 Foraging optimally in social neuroscience: computations and methodological considerations. Soc. Cogn. Affect. Neurosci. 30, nsaa037. (doi:10.1093/scan/nsaa037) Crossref, Google Scholar  54.
Iwasa Y, Higashi M, Yamamura N . 1981 Prey distribution as a factor determining the choice of optimal foraging strategy. Am. Nat. 117, 710723. (doi:10.1086/283754) Crossref, Web of Science, Google Scholar  55.
Gold JI, Shadlen MN . 2007 The neural basis of decision making. Annu. Rev. Neurosci. 30, 535574. (doi:10.1146/annurev.neuro.29.051605.113038) Crossref, PubMed, Web of Science, Google Scholar  56.
Brunton BW, Botvinick MM, Brody CD . 2013 Rats and humans can optimally accumulate evidence for decisionmaking. Science 340, 9598. (doi:10.1126/science.1233912) Crossref, PubMed, Web of Science, Google Scholar  57.
Gardiner CW . 2004 Handbook of stochastic methods, vol. 3. New York, NY: Springer. Crossref, Google Scholar  58.
Kilpatrick ZP, Davidson JD, Hady AE . 2020 Normative theory of patch foraging decisions. (http://arxiv.org/abs/2004.10671). Google Scholar  59.
Olsson O . 2006 The foraging benefits of information and the penalty of ignorance. Oikos 112, 260273. (doi:10.1111/j.00301299.2006.13548.x) Crossref, Web of Science, Google Scholar  60.
Dall SR, Giraldeau LA, Olsson O, McNamara JM, Stephens DW . 2005 Information and its use by animals in evolutionary ecology. Trends Ecol. Evol. 20, 187193. (doi:10.1016/j.tree.2005.01.010) Crossref, PubMed, Web of Science, Google Scholar  61.
Glaze CM, Filipowicz AL, Kable JW, Balasubramanian V, Gold JI . 2018 A bias–variance tradeoff governs individual differences in online learning in an unpredictable environment. Nat. Hum. Behav. 2, 213224. (doi:10.1038/s4156201802974) Crossref, Web of Science, Google Scholar  62.
Stephens DW, Brown JS, Ydenberg RC . 2008 Foraging: behavior and ecology. Chicago, IL: University of Chicago Press. Google Scholar  63.
Cuthill IC, Kacelnik A, Krebs JR, Haccou P, Iwasa Y . 1990 Starlings exploiting patches: the effect of recent experience on foraging decisions. Anim. Behav. 40, 625640. (doi:10.1016/S00033472(05)80692X) Crossref, Web of Science, Google Scholar  64.
Van Gils JA, Schenk IW, Bos O, Piersma T . 2003 Incompletely informed shorebirds that face a digestive constraint maximize net energy gain when exploiting patches. Am. Nat. 161, 777793. (doi:10.1086/374205) Crossref, PubMed, Web of Science, Google Scholar  65.
Drugowitsch J, MorenoBote R, Pouget A . 2014 Optimal decisionmaking with timevarying evidence reliability. In Advances in neural information processing systems (eds Z Ghahramani, M Welling, C Cortes, N Lawrence, KQ Weinberger), pp. 748–756. Red Hook, NY: Curran Associates, Inc. Google Scholar  66.
Ratcliff R, McKoon G . 2007 The diffusion decision model: theory and data for twochoice decision tasks. Neural Comput. 20, 873922. (doi:10.1162/neco.2008.1206420) Crossref, Web of Science, Google Scholar  67.
Morris DW . 2003 Toward an ecological synthesis: a case for habitat selection. Oecologia 136, 113. (doi:10.1007/s0044200312414) Crossref, PubMed, Web of Science, Google Scholar  68.
Nishimura K . 1992 Foraging in an uncertain environment: patch exploitation. J. Theor. Biol. 156, 91111. (doi:10.1016/S00225193(05)806587) Crossref, Web of Science, Google Scholar  69.
Klaassen RH, Nolet BA, van Gils JA, Bauer S . 2006 Optimal movement between patches under incomplete information about the spatial distribution of food items. Theor. Popul. Biol. 70, 452463. (doi:10.1016/j.tpb.2006.04.002) Crossref, PubMed, Web of Science, Google Scholar  70.
Hughes JJ, Ward D, Perrin MR . 1995 Effects of substrate on foraging decisions by a Namib Desert gerbil. J. Mammal. 76, 638645. (doi:10.2307/1382372) Crossref, Web of Science, Google Scholar  71.
Valone TJ . 1991 Bayesian and prescient assessment: foraging with preharvest information. Anim. Behav. 41, 569577. (doi:10.1016/S00033472(05)808978) Crossref, Web of Science, Google Scholar  72.
Marschall EA, Chesson PL, Stein RA . 1989 Foraging in a patchy environment: preyencounter rate and residence time distributions. Anim. Behav. 37, 444454. (doi:10.1016/00033472(89)900912) Crossref, Web of Science, Google Scholar  73.
Kacelnik A, Bateson M . 1996 Risky theories—the effects of variance on foraging decisions. Am. Zool. 36, 402434. (doi:10.1093/icb/36.4.402) Crossref, Google Scholar  74.
Cisek P, Puskas GA, ElMurr S . 2009 Decisions in changing conditions: the urgencygating model. J. Neurosci. 29, 11 56011 571. (doi:10.1523/JNEUROSCI.184409.2009) Crossref, Web of Science, Google Scholar  75.
Drugowitsch J, MorenoBote R, Churchland AK, Shadlen MN, Pouget A . 2012 The cost of accumulating evidence in perceptual decision making. J. Neurosci. 32, 36123628. (doi:10.1523/JNEUROSCI.401011.2012) Crossref, PubMed, Web of Science, Google Scholar  76.
Malhotra G, Leslie DS, Ludwig CJ, Bogacz R . 2018 Timevarying decision boundaries: insights from optimality analysis. Psychon. Bull. Rev. 25, 971996. (doi:10.3758/s1342301713406) Crossref, PubMed, Web of Science, Google Scholar  77.
Nathan R . 2008 An emerging movement ecology paradigm. Proc. Natl Acad. Sci. USA 105, 19 05019 051. (doi:10.1073/pnas.0808918105) Crossref, Web of Science, Google Scholar  78.
Jackson BJ, Fatima GL, Oh S, Gire DH . 2019 Many paths to the same goal: metaheuristic operation of brains during natural behavior. bioRxiv 697607. (doi:10.1101/697607) Google Scholar  79.
Wikenheiser AM, Stephens DW, Redish AD . 2013 Subjective costs drive overly patient foraging strategies in rats on an intertemporal foraging task. Proc. Natl Acad. Sci. USA 110, 83088313. (doi:10.1073/pnas.1220738110) Crossref, PubMed, Web of Science, Google Scholar  80.
Greene CH . 1986 Patterns of prey selection: implications of predator foraging tactics. Am. Nat. 128, 824839. (doi:10.1086/284608) Crossref, Web of Science, Google Scholar  81.
Giraldeau LA, Caraco T . 2018 Social foraging theory, vol. 73. Princeton, NJ: Princeton University Press. Crossref, Google Scholar  82.
Harper DGC . 1982 Competitive foraging in mallards: ‘ideal free’ ducks. Anim. Behav. 30, 575584. (doi:10.1016/S00033472(82)800717) Crossref, Web of Science, Google Scholar  83.
Templeton JJ, Giraldeau LA . 1996 Vicarious sampling: the use of personal and public information by starlings foraging in a simple patchy environment. Behav. Ecol. Sociobiol. 38, 105114. (doi:10.1007/s002650050223) Crossref, Web of Science, Google Scholar  84.
PérezEscudero A, Polavieja GGD . 2011 Collective animal behavior from Bayesian estimation and probability matching. PLoS Comput. Biol. 7, e1002282. (doi:10.1371/journal.pcbi.1002282) Crossref, PubMed, Web of Science, Google Scholar  85.
Hein AM, Altshuler DL, Cade DE, Liao JC, Martin BT, Taylor GK . 2020 An algorithmic approach to natural behavior. Curr. Biol. 30, R663R675. (doi:10.1016/j.cub.2020.04.018) Crossref, PubMed, Web of Science, Google Scholar