Dependency in multisensory integration: a copula-based analysis

1. Introduction: copulas

The theory of copulas has stirred a lot of interest in recent years in several areas of statistics, including actuarial science and finance (e.g. [1]). More recently, it has also captured the attention of researchers in contextuality and probability. The main reasons for this development are the following properties: copulas allow one (i) to study the structure of stochastic dependency in a ‘scale-free’ manner, i.e. independent of the specific marginal distributions and (ii) to construct families of multivariate distributions with specified properties. Briefly, a copula is a function that joins a multivariate distribution to its one-dimensional marginal distribution functions. A formal definition for any finite dimension n is the following.

Definition 1.1.

A function $\mathbf{C}:{[0,1]}^n\to [0,1]$ is called n-dimensional copula if there is a probability space $(\mathit{\Omega },\mathcal{F},\mathbb{P})$ supporting a vector of standard uniform random variables (U₁, …, U_n) such that

$$\mathbf{C}(u_1,\dots ,u_n)=P(U_1\le u_1,\dots ,U_n\le u_n),u_1,\dots ,u_n\in [0,1].$$

The following theorem by Sklar [2] laid the foundation of many subsequent studies (for a proof, e.g. [3]).

Theorem 1.2 (Sklar's theorem, 1959).

LetF(x₁, …, x_n) be ann-variate distribution function with marginsF₁(x₁), …, F_n(x_n); then there exists ann-copulaC:[0, 1]ⁿ→[0, 1] that satisfies

$$F(x_1,\dots ,x_n)=\mathbf{C}(F_1(x_1),\dots ,F_n(x_n)),(x_1,\dots ,x_n)\in {\mathrm{\Re }}^n.$$

If all univariate marginsF₁, …, F_nare continuous, then the copula is unique. IfF⁻¹₁, …, F⁻¹_nare the quantile functions of the margins, then for any (u₁, …, u_n)∈[0, 1]ⁿ

$$\mathbf{C}(u_1,\dots ,u_n)=F(F_1^{-1}(u_1),\dots ,F_n^{-1}(u_n)).$$

Here, we demonstrate the use of copulas by investigating the structure of dependency between processing stages in a stochastic model of reaction time (RT) for multisensory integration. Next, we briefly present the time-window-of-integration (TWIN) model developed to describe the effect of multisensory stimulation on RT [4]. Then, we consider the model from the perspective of copula theory, limited to the case of n = 2, and derive explicit terms for the covariance and Kendall's tau between the processing stages without making assumptions about parametric distribution families. Moreover, we point out the specific role played by two stochastic order relations, the so-called usual stochastic order and the likelihood ratio order, in determining the sign of dependency.

2. Time-window-of-integration model

When stimulus information, perceived via several sensory modalities, indicates the occurrence of some event, an observer is typically faster responding to the stimulus complex compared to receiving only unimodal information. For example, when a flash occurring randomly to the left or right of a fixation point is presented together with an auditory stimulus (e.g. click) presented in a close temporal proximity, the time to start moving the eyes towards the stimulus location (i.e. saccadic RT) is typically reduced by 40–100 ms, depending on the specific experimental set-up. This is considered as an instantiation of multisensory integration, a topic that has attracted a lot of attention from researchers in psychology and neuroscience [5]. The occurrence of multisensory integration (MI) critically depends on the temporal arrangement of the stimulus components as well. Specifically, the speed-up of reaction time to a visual–auditory stimulus occurs only if stimuli from both modalities are registered by the sensory system within a certain time interval (aka ‘time window of integration’) and typically becomes greatest when a visual stimulus precedes an auditory by an interval that equals the difference in RT between response to the visual alone and the auditory alone (e.g. [6,7]). In the time-window-of-integration model of MI [4], RT is assumed to be some combination of random variables representing various sub-processing times defined with respect to a common probability space. Specifically, we let V and A denote processing times for visual and auditory stimuli, respectively, in a first peripheral processing stage followed by a second, more central stage. Let (W₁, W₂) be the random vector with W₁ presenting some function of A and V, e.g. $min(A,V)$ in the ‘redundant-signals task’ [6], and W₂ denoting the random duration of the second stage that includes central processing like stimulus identification and response preparation. The purpose of this note is to investigate the effect of a time window on the stochastic dependency between W₁ and W₂ at a rather general level. Observable RT in the auditory–visual condition is taken as

The time-window assumption holds that the occurrence of multisensory integration depends on V and A terminating sufficiently close in time. Let I denote the event that this happens, i.e. with ω denoting the ‘width’ of the time window, assumed to be a positive constant (we write I(ω) = I if no confusion arises). W₁ and W₂ are defined on the same probability space but they are not necessarily statistically independent. With $\pi =\mathbb{P}[I]$, expected RT in the auditory–visual condition can then be decomposed as follows: where $\mathbb{E}[W_i|I]$ and $\mathbb{E}[W_i|C]$ (i = 1, 2) denote expected processing times in the i-th stage conditioned on I and its complement, C, respectively. Setting for the magnitude of the integration effect in the i-th stage results in The term π × (Δ₁ + Δ₂) is interpreted as a measure of the expected combined MI effect in both stages, with positive values corresponding to facilitation, negative ones to inhibition. An important assumption often made in the TWIN model is that π and (Δ₁ + Δ₂) can be estimated independently by varying certain conditions of the experimental context (for details, e.g. [8,9]). Up to now, both mathematical analysis and empirical testing of the model have mostly been limited to the means (i.e. expectation) of the random processing times. For further insight, it is worthwhile to extend the analysis to higher moments, like variance of T and covariance between W₁ and W₂: where $\mathbb{C}$ stands for the covariance. The TWIN model assumes that W₁ and W₂ are conditionally independent, with conditioning on either I or its complement, C. Thus, in general, $\mathbb{C}[W_1,W_2]$ may be different from zero. Obviously, explicit expressions for $\mathbb{C}[W_1,W_2]$ are only obtainable if specific assumptions about the distribution of W₁ and W₂ are added like, e.g., ex-Gaussian [9]. The goal pursued here is to characterize the dependency between W₁ and W₂ without assuming that the random variables follow some specified parametric family of distributions. First, linear dependence will be studied, followed by characterizing stochastic dependency at a more general level with the help of concepts from copula theory.

3. Linear dependency

(a) Conditional independence and mixture distribution

We introduce the bivariate distribution function for the non-negative random vector (W₁, W₂) of processing times:

with marginal distributions H(w₁, ∞) = H₁(w₁) and H(∞, w₂) = H₂(w₂). Conditioning on the events I and C, this distribution is expressed as a bivariate mixture, where H_I and H_C denote the conditional distributions of W₁ and W₂ with respect to I and C, respectively. By conditional independence, H_I and H_C can be written as products of their marginal distributions, where F and G refer to the first and second stage (conditional) distributions, respectively. Inserting these into equation (3.2) yields The marginal distributions for H are From now on, all distribution functions are assumed to be absolutely continuous with strictly positive densities and to have finite first and second moments.

(b) Covariance of W₁ and W₂

For the reader's convenience, we list three useful properties about conditional second moments. Let X, Y, Z be random variables defined on some probability space. Conditional covariance, assuming all moments exist, is defined as

Setting X = Y yields the conditional variance It can be shown that [10]

(a)	${\mathbb{E}}_Z[\mathbb{E}[X|Z]]=\mathbb{E}[X]$ (‘law of total expectation’);
(b)	$\mathbb{C}[X,Y]=\mathbb{C}[{\mathbb{E}}_Z[X|Z],{\mathbb{E}}_Z[Y|Z]]+{\mathbb{E}}_Z[\mathbb{C}[X,Y|Z]]$ (‘law of total covariance’);
(c)	$\mathbb{V}[X]={\mathbb{E}}_Z[\mathbb{V}[X|Z]]+{\mathbb{V}}_Z[\mathbb{E}[X|Z]]$ (‘conditional variance’ formula).

The subscripts of $\mathbb{E}$ and $\mathbb{V}$ are dropped if no confusion arises. We define Z as an indicator function for the ‘integration’ event:

Covariance between W₁ and W₂ is then evaluated using the above ‘law of total covariance’ (b): Hence, after some algebra, covariance can be written as where, for simpler notation, we have introduced the following abbreviations: Given π is different from zero or one, covariance between W₁ and W₂ is positive if and only if for both variables the mean is (strictly) larger, or smaller, under the event of ‘no integration’ than under ‘integration’. If the means are equal for W₁, or W₂, or both, covariance becomes zero. Finally, covariance is negative if the ordering of means with respect to ‘integration/no integration’ differs between the two variables.

Note that an alternative way to obtain the covariance is to appeal to Hoeffding's lemma (see below), by inserting the distributions and rearranging them.

(c) From local to global dependency

Although we have just characterized the sign of the covariance between W₁ and W₂ in terms of conditional expectations, one may also ask about properties of the conditional distributions F_I, G_I, F_C, G_C that (i) are defined locally and (ii) are sufficient to determine the direction of dependence.

The key tool is to refer to the well-known Hoeffding's lemma [11] formulated here for (W₁, W₂).

Lemma 3.1 (Hoeffding).

$$\mathbb{C}[W_1,W_2]={\iint }_0^{\mathrm{\infty }}[H(w_1,w_2)-H_1(w_1)H_2(w_2)]\hspace{0.17em}\mathrm{d}{w}_1\hspace{0.17em}\mathrm{d}{w}_2.$$

It turns out that the usual stochastic order relation suffices to determine the direction of dependence between W₁ and W₂.

Definition 3.2.

Let X and Y be two random variables such that

$$\mathbb{P}(X>x)\le \mathbb{P}(Y>x)\text{ for all }x\in (\hspace{0.17em}-\hspace{0.17em}\mathrm{\infty },\hspace{0.17em}\mathrm{\infty }).$$

Then X is said to be smaller than Y in the usual stochastic order, denoted as

$$X{\le }_{\mathit{s}\mathit{t}}Y.$$

Theorem 3.3.

LetF_C(w₁) > F_I(w₁) andG_C(w₂) > G_I(w₂) for allw₁, w₂∈[0, ∞); then, $\mathbb{C}[W_1,W_2]$is positive. If both order relations are reversed, covariance is positive again. Reversing only one of the signs implies the covariance to be negative.

Proof.

In lemma 3.1 replace H, H₁ and H₂ according to equations (3.3) and (3.4); after algebraic rearrangement, the integrand becomes

$$F_C(w_1)G_C(w_2)-F_I(w_1)G_C(w_2)-F_C(w_1)G_I(w_2)+F_I(w_1)G_I(w_2);$$

thus, the covariance is positive if

$$[F_C(w_1)-F_I(w_1)][G_C(w_2)-G_I(w_2)]>0;$$

the theorem follows by reversing these steps; the case of negative covariance is analogous. ▪

To summarize, assuming that the distribution of W_i under integration (respectively, no integration) strictly dominates the distribution of W_i under no integration (resp. integration) relative to the usual stochastic order relation ≤_st, for i = 1, 2, implies positive (resp. negative) covariance between W₁ and W₂.

(d) Variance

For completeness, the corresponding equations for the variances of W₁ and W₂ are presented next. They are derived using the expression for a conditional variance. For i = 1, 2,

The last identity reveals that the variance of both the first and second processing stage is composed of a weighted average of the conditional variances (weighted by the probability of I and C occurring) plus an additional, non-negative term π(1 − π)Δ²_i. The latter represents the part of variability that is due to the mixture generated by the occurrence of I or C. Obviously, this effect will be maximal for π = 0.5.

4. Nonlinear dependency

By Sklar's theorem [2], there exists a copula C such that

and, assuming the marginals are continuous, C is unique. It is also known that a binary mixture of copulas is again a copula [3]. However, although H is a mixture of product copulas, it need not be a product copula itself, as this example demonstrates.

Example 4.1 (D. Pfeifer 2013, personal communication).

$$H(x,y)=\frac{xy}{2}(x+y)=\frac{1}{2}x{y}^2+\frac{1}{2}{x}^{2}y=\frac{1}{2}{F}_1(x)G_1(y)+\frac{1}{2}{F}_2(x)G_2(y),$$

with F₁(x) = x, F₂(x) = x², G₁(y) = y², G₂(y) = y for 0≤x, y≤1. Thus, H is a mixture of product distribution functions. For the marginals,

$$H_1(x)=H(x,1)=\frac{1}{2}x(x+1)\mathrm{and}{H}_2(y)=H(1,y)=\frac{1}{2}y(y+1),$$

for 0≤x, y≤1. However, for copula C one has

$$\begin{array}{rl}\mathbf{C}(u,v)& =H(H_1^{-1}(u),H_2^{-1}(v))=H(-\frac{1}{2}+\frac{1}{2}\sqrt{1+8u},-\frac{1}{2}+\frac{1}{2}\sqrt{1+8v})\\ \hspace{0.17em}& =\frac{1}{16}(-4+\sqrt{1+8u}+\sqrt{1+8v})(-1+\sqrt{1+8u})(-1+\sqrt{1+8v})\end{array}$$

for 0≤u, v≤1, which is not a mixture of product copulas.

Inserting the expression for the marginals as mixtures, equation (4.1) can be written as

With H_i(w_i) = u_i, i = 1, 2, the copula is In contrast to example 4.1, here the inverse functions can, in general, not be obtained explicitly. Rather than introducing specific distributional families for F_I, G_I, F_C, G_C, we will study a general measure of dependency that extends our results for linear dependency.

(a) Kendall's tau for TWIN model

Let (W_i1, W_i2), i = 1, 2, be two independent and identically distributed vectors with joint distribution function H and copula C. At the population level, Kendall's tau is defined as the probability of ‘concordance’ minus the probability of ‘discordance’ (e.g. [1,3]), i.e.

Since $\mathbb{P}[(W_{11}-W_{21})(W_{12}-W_{22})<0)]=1-\mathbb{P}[(W_{11}-W_{21})(W_{12}-W_{22})>0)],$

Theorem 4.2.

For the random processing times inTWIN, Kendall's tau equals

$$\tau (W_1,W_2)=2\pi (1-\pi )(2V-1),$$4.7

with

$$V=\int F_C(w_1)\hspace{0.17em}\mathrm{d}{F}_I(w_1)\int G_C(w_2)\hspace{0.17em}\mathrm{d}{G}_I(w_2)+\int F_I(w_1)\hspace{0.17em}\mathrm{d}{F}_C(w_1)\int G_I(w_2)\hspace{0.17em}\mathrm{d}{G}_C(w_2),$$

where integration is over the positive reals, ℜ+.

Proof.

Given that

$$\mathbb{P}[(W_{11}-W_{21})(W_{12}-W_{22})>0]=\mathbb{P}[W_{11}>W_{21},W_{12}>W_{22}]+\mathbb{P}[W_{11}<W_{21},W_{12}<W_{22}],$$

these probabilities can be evaluated by integrating over the distribution of one of the vectors, (W₁₁, W₁₂) or (W₂₁, W₂₂). For example,

$$\begin{array}{rl}\mathbb{P}[W_{11}>W_{21},W_{12}>W_{22}]& =\mathbb{P}[W_{21}<W_{11},W_{22}<W_{12}]\\ \hspace{0.17em}& ={\iint }_{{\mathrm{\Re }}_{+}^2}\mathbb{P}[W_{21}\le w_1,W_{22}\le w_2]\hspace{0.17em}\mathrm{d}H(w_1,w_2)\\ \hspace{0.17em}& ={\iint }_{{\mathrm{\Re }}_{+}^2}\mathbb{P}[W_{21}\le w_1,W_{22}\le w_2]\hspace{0.17em}\mathrm{d}\mathbf{C}(H_1(w_1),H_2(w_2))\\ \hspace{0.17em}& ={\iint }_{{\mathrm{\Re }}_{+}^2}H(w_1,w_2)\hspace{0.17em}\frac{{\mathrm{\partial }}^{2}H(w_1,w_2)}{\mathrm{\partial }{w}_1\mathrm{\partial }{w}_2}\hspace{0.17em}\mathrm{d}{w}_1\hspace{0.17em}\mathrm{d}{w}_2.\end{array}$$4.8

Let f_I(w₁), g_I(w₂), f_C(w₁), g_C(w₂) denote the densities corresponding to F_I(w₁), G_I(w₂), F_C(w₁), G_C(w₂), respectively. Then,

$$\frac{{\mathrm{\partial }}^{2}H(w_1,w_2)}{\mathrm{\partial }{w}_1\mathrm{\partial }{w}_2}=h(w_1,w_2)=\pi f_I(w_1)g_I(w_2)+(1-\pi )f_C(w_1)g_C(w_2).$$4.9

Inserting explicit expressions for H(w₁, w₂) and its second partial derivative into equation (4.8) results in

$$\begin{array}{rl}\mathbb{P}[W_{11}>W_{21},W_{12}>W_{22}]& ={\iint }_{{\mathrm{\Re }}_{+}^2}[\pi F_I(w_1)G_I(w_2)+(1-\pi )F_C(w_1)G_C(w_2)]\\ \hspace{0.17em}& \times [\pi f_I(w_1)g_I(w_2)+(1-\pi )f_C(w_1)g_C(w_2)]\hspace{0.17em}\mathrm{d}{w}_1\hspace{0.17em}\mathrm{d}{w}_2.\end{array}$$4.10

Dropping the arguments w₁ and w₂ for simpler notation, expanding terms under the integral and rearranging them yields for the integrand over w₁ and w₂:

$$\begin{array}{rl}\hspace{0.17em}& F_C\times f_C\times G_C\times g_C\\ \hspace{0.17em}& +\pi \hspace{0.17em}(-2F_C\times f_C\times G_C\times g_C+F_C\times f_I\times G_C\times g_I\hspace{0.17em}+F_I\times f_C\times G_I\times g_C)\\ \hspace{0.17em}& +{\pi }^2(F_C\times f_C\times G_C\times g_C-F_C\times f_I\times G_C\times g_I-F_I\times f_C\times G_I\times g_C+F_I\times f_I\times G_I\times g_I).\end{array}$$

Observing that, e.g., the integral over F_C × f_C × G_C × g_C amounts to

$${\int }_0^{\mathrm{\infty }}({\int }_0^{\mathrm{\infty }}{F}_C(w_1)f_C(w_1)\hspace{0.17em}\mathrm{d}{w}_1)G_C(w_2)g_C(w_2)\hspace{0.17em}\mathrm{d}{w}_2=\frac{1}{2}\times \frac{1}{2}=\frac{1}{4},$$

the integral of equation (4.10) results in

$$\mathbb{P}[W_{11}>W_{21},W_{12}>W_{22}]=\frac{1}{4}-\frac{\pi }{2}+\pi V+{\pi }^2(\frac{1}{4}-V+\frac{1}{4}),$$4.11

where

$$V\equiv {\iint }_{{\mathrm{\Re }}_{+}^2}[F_C\times f_I\times G_C\times g_I+F_I\times f_C\times G_I\times g_C]\hspace{0.17em}\mathrm{d}{w}_1\hspace{0.17em}\mathrm{d}{w}_2.$$4.12

Because of symmetry, $\mathbb{P}[W_{11}>W_{21},W_{12}>W_{22}]=\mathbb{P}[W_{11}<W_{21},W_{12}<W_{22}]$, so that

$$\begin{array}{rl}\tau (W_1,W_2)& =4\times (\frac{1}{4}-\frac{\pi }{2}+\pi V+{\pi }^2(\frac{1}{4}-V+\frac{1}{4}))-1\\ \hspace{0.17em}& =2\pi (1-\pi )(2V-1).\end{array}$$

 ▪

Corollary 4.3.

Assumeπis different from zero or one; then

(i)	IfFC = FIorGC = GI, thenτ(W1, W2) = 0;
(ii)	τ(W1, W2)≥ − (1/2).

Proof.

For (i): Assume F_C = F_I; then,

$$\begin{array}{rl}V& ={\int }_0^{\mathrm{\infty }}({\int }_0^{\mathrm{\infty }}{F}_C(w_1)f_C(w_1)\hspace{0.17em}\mathrm{d}{w}_1)[G_C(w_2)g_I(w_2)+G_I(w_2)g_C(w_2)]\mathrm{d}{w}_2\\ \hspace{0.17em}& =\frac{1}{2}.\end{array}$$

The case G_C = G_I follows by symmetry. For (ii): as an integral over a linear combination of non-negative functions, V is non-negative. Thus, the smallest value for τ is obtained by setting V = 0 and π = (1/2). ▪

The corollary shows that Kendall's τ will be non-zero only if both pairs of marginal distributions, (F_I, F_C) or (G_I, G_C), contain non-identical distributions; thus, for τ to be non-zero, there must be an effect of integration on the distribution in both first and second stage processing. In contrast, for covariance $\mathbb{C}[W_1,W_2]$ (equation (3.9)) to be non-zero there must be an effect of integration on the expected values in both pairs of marginal distributions, i.e. in both the first and second stage. In other words if e.g. (F_I, F_C) have equal means but different variances, processing times W₁ and W₂ will be linearly independent but may exhibit nonlinear dependency (nonzero τ).

(b) From local to global dependency

As shown in the last section, Kendall's tau for the TWIN model may be positive or negative. In analogy to the linear dependence, the question arises whether there exists an order relation on the random variables that determines the sign of dependency. The answer is in the affirmative: it is again a well-known stochastic order relation (e.g. [12], p. 42)

Definition 4.4.

Let X and Y be continuous random variables with densities f and g such that

$$\frac{g(t)}{f(t)}\text{increases in }t\text{ over the union of support of }X\text{ and }Y,$$

(here, a/0 is taken to be equal to ∞ whenever a > 0) or, equivalently,

$$f(x)g(y)\ge f(y)g(x)\text{for all }x\le y.$$

Then X is said to be smaller than Y in the likelihood ratio order, denoted as

$$X{\le }_{\mathit{l}\mathit{r}}Y.$$

The key here is the following representation of Kendall's tau by Nelsen [13]:

Lemma 4.5 ([13]).

LetXandYbe continuous random variables with joint density functiong(x, y); with −∞ < x₁ < x₂ < ∞ and −∞ < y₁ < y₂ < ∞

$$\tau (X,Y)=2{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{y_2}{\int }_{-\mathrm{\infty }}^{x_2}[g(x_2,y_2)g(x_1,y_1)-g(x_1,y_2)g(x_2,y_1)]\hspace{0.17em}\mathrm{d}{x}_1\hspace{0.17em}\mathrm{d}{y}_1\hspace{0.17em}\mathrm{d}{x}_2\hspace{0.17em}\mathrm{d}{y}_2.$$4.13

Note that function g is called positively likelihood ratio-dependent (PLR), or totally positive of order 2 (TP₂) if the above integrand is non-negative. With these preparations, we state the following theorem for TWIN model random variables. Note that we write e.g.

for the likelihood ratio order defined by the densities f_C(w₁), f_I(w₁) and g_C(w₂), g_I(w₂), respectively.

Theorem 4.6.

Let [W₁|I]≤_lr[W₁|C] and [W₂|I]≤_lr[W₂|C]; thenτ(W₁, W₂)≥0. If both order relations are reversed, thenτ(W₁, W₂)≥0 again. If only one of the order relations is reversed, thenτ(W₁, W₂)≤0.

Proof.

Consider the integrand of equation (4.13) with g(x, y) replaced by the joint density h(w₁, w₂) of (W₁, W₂). For w₁≤w₁′ and w₂≤w₂′, the integrand becomes

$$h(w_1^{\prime },w_2^{\prime })\hspace{0.17em}h(w_1,w_2)-h(w_1,w_2^{\prime })\hspace{0.17em}h(w_1^{\prime },w_2).$$

Inserting according to equation (4.9),

$$h(w_1,w_2)=\pi f_I(w_1)g_I(w_2)+(1-\pi )f_C(w_1)g_C(w_2),$$

yields for the integrand, after routine algebraic manipulation,

$$\pi \times (1-\pi )\times [f_C(w_1^{\prime })f_I(w_1)-f_C(w_1)f_I(w_1^{\prime })]\times [g_C(w_2)g_I(w_2^{\prime })-g_C(w_2^{\prime })g_I(w_2)].$$

The terms in brackets will be non-negative if both

$$\frac{f_C(w_1^{\prime })}{f_I(w_1^{\prime })}\ge \frac{f_C(w_1)}{f_I(w_1)}\text{ and }\frac{g_C(w_2^{\prime })}{g_I(w_2^{\prime })}\ge \frac{g_C(w_2)}{g_I(w_2)},$$

that is, [W₁|I]≤_lr[W₁|C] and [W₂|I]≤_lr[W₂|C]. Thus, the integrand will be non-negative and τ(W₁, W₂)≥0. The other claims of the theorem follow trivially. ▪

5. Conclusion

According to the TWIN model, RT is an additive combination of two random variables, the first and second stage processing times W₁ and W₂ with the joint distribution being a binary mixture with indicator function Z for the event of integration (see equation (3.7)). We derived explicit terms for two measures of dependency between W₁ and W₂, covariance and Kendall's tau. Specifically, we showed that Kendall's tau may not be smaller than −1/2 and will equal zero whenever there is no effect of integration in either the first or second stage. Moreover, comparing processing times under integration with those without integration with respect to two stochastic order relations—the usual stochastic order and the likelihood ratio order—determines the sign of dependency.

It is noteworthy that our statistical results do not at all depend on the exact definition of event I. It has to be a binary event but can be arbitrary otherwise. Obviously, P(I) = π is an increasing function of window width parameter ω. Because of the factor π(1 − π) occurring in both terms for the dependency between W₁ and W₂, this implies that its strength will be maximal for a value of ω yielding π = 0.5.

As mentioned in the introductory section, numerical values for the covariance or Kendall's tau cannot be obtained without explicit assumptions about the distribution of W₁ and W₂. The simple reason is that these random variables are not observable, only there sum is, according to the model. One may thus wonder whether any of the results derived here will allow empirical testing of some aspects of the model.

We will not cover this issue in full depth here because it would require presenting more of the current state of research in multisensory integration. However, our results are likely to be relevant in the context of some controversy about the role of variability, specifically comparing the variance of RT to unimodal versus multisensory stimuli. First, from equation (2.4), it is clear that, in TWIN, the covariance between W₁ and W₂ directly modulates the (observable) variance of their sum. Moreover, this covariance is a function of the probability of integration π, which in turn depends on the size of the time window of integration ω. There are a number of well-tested experimental manipulations to influence the time window (e.g. [8,9]), so this will allow deriving certain hypotheses about RT variance in the context of additional assumptions. Second, it is known that the usual stochastic order is closed under convolution for (conditionally) independent random variables; then, according to theorem 3.3, the covariance will be positive/negative under a strict stochastic ordering of the response times under integration versus no integration. An analogous observation holds for the likelihood ratio order and Kendall's tau with the additional requirement of logconcave densities (e.g. [12]). Thus, for certain broad families of distributions, further predictions about RT variability can be derived. This general route of investigation seems promising but needs further scrutiny.

Data accessibility

This article has no additional data.

Authors' contributions

H.C. and A.D. conceived of the study, and H.C. drafted the manuscript. Both authors read and approved the manuscript.

Competing interests

We declare we have no competing interests.

Funding

A.D. was supported by grant no. DI 506/12-1 (German Science Foundation/DFG). H.C. was supported by a grant from German Science Foundation/DFG (SFB/TRR31) and Cluster of Excellence (DFG) ‘Hearing4all’ of Oldenburg University.

Footnotes

One contribution of 16 to a theme issue ‘Contextuality and probability in quantum mechanics and beyond’.