Electronic supplementary material: Hebbian plasticity requires compensatory processes on multiple timescales

We review a body of theoretical and experimental research on Hebbian and homeostatic plasticity, starting from a puzzling observation: while homeostasis of synapses found in experiments is a slow compensatory process, most mathematical models of synaptic plasticity use rapid compensatory processes (RCPs). Even worse, with the slow homeostatic plasticity reported in experiments, simulations of existing plasticity models cannot maintain network stability unless further control mechanisms are implemented. To solve this paradox, we suggest that in addition to slow forms of homeostatic plasticity there are RCPs which stabilize synaptic plasticity on short timescales. These rapid processes may include heterosynaptic depression triggered by episodes of high postsynaptic firing rate. While slower forms of homeostatic plasticity are not sufficient to stabilize Hebbian plasticity, they are important for fine-tuning neural circuits. Taken together we suggest that learning and memory rely on an intricate interplay of diverse plasticity mechanisms on different timescales which jointly ensure stability and plasticity of neural circuits. This article is part of the themed issue ‘Integrating Hebbian and homeostatic plasticity’.


S1 Mathematical digression: The notion of a timescale
To formally introduce the notion of a timescale, we consider a linear system defined by the differential equation τ dy dt = −(y − ξ) (S1) where ξ and τ are two fixed positive parameters. The solution y(t) of the differential equation relaxes exponentially to the value y = ξ, on the timescale τ . Practically speaking that means that no matter how we initialize y(t) at time t = 0, if we wait for a time 3τ , the difference y(t) − ξ will have decreased by 95%. The exponential evolution is typical for a linear system. Moreover, the equation has a nice scaling behavior: If we know the solution for a parameter setting τ = 1, we can get the solution for the parameter setting τ = 5 by multiplying all times by a factor of five. This is the reason why we speak of τ as the timescale of the system.
The fact that a constant value of y(t) = ξ for all t is a solution means that ξ is a fixed point of the system. To check this, remember that the derivative of a constant is zero (so that the left-hand side of Eq. (S1) vanishes). And for y = ξ the right-hand side is obviously zero, too. Moreover, this fixed point is stable because from any initial condition y(t) will converge towards ξ.
Let us now consider the case in which τ is negative. In this case, y(t) = ξ is still a fixed point, but if we start with a value y(t) > ξ, then y(t) will explode exponentially fast to large positive values while for an initial value y(t) < ξ it will explode exponentially fast to large negative values.
The timescale of explosion is again given by τ .
However, our intuition for timescales breaks down for nonlinear systems. Therefore, when we consider a nonlinear system τ dy dt = f (y), it does not suffice for τ to be large to be able to talk about a slow or fast timescale. We have to be specific about the behavior of f in the regime that we are interested in. The mathematical trick to do this is to look for a fixed point of the equation, that is a value of y with f (y) = 0. Suppose y = y 0 is a fixed point. We then study the derivative df /dy at Another example of a linear system is a low-pass filter. Suppose we pass our variable y(t) through a low-pass filter with time constant τ d to yield By taking the derivative on both sides of Eq. (S2), we find that the low-pass filterȳ is the solution . This equation is similar to equation (S1), if we replace the constant target value ξ by a time-dependent target y(t)/τ d .

S2 The induction timescale of Hebbian plasticity
Because both learning rules and neurons are typically nonlinear, the mathematical definition of a timescale for Hebbian plasticity requires the existence of a fixed point. The reason is that for a nonlinear system we can define a timescale only in the vicinity of a fixed point, as we have seen above.
Even if a plasticity rule in isolation looks linear, the combination of plasticity with the neuronal dynamics typically makes the network as a whole nonlinear. Consider for instance a single neuron i that is driven by N inputs arriving at synapses w ij for 1 ≤ j ≤ N . In a rate model, the state of the postsynaptic neuron is characterized by its firing rate y i = g N j=1 w ij x j where x j is the firing rate of the presynaptic neuron with index j. The function g denotes the frequency-current relation of a single neuron and we assume that it is monotonically increasing, i.e., if we increase the input, the firing rate increases as well. In the theoretical literature g is sometimes called the gain function of the neuron, hence our choice of letter g.
Let us first study a simple Hebbian learning rule Note that this is a special instance within the general framework of Eqs. (1,2) in the main manuscript.
To see this, consider a Hebbian term H(post i , pre j ) = H(y i , x j ) = y i x j , and a 1 (w ij ) = η and set all other terms in Eq. (2) to zero. This learning rule is linear in x j and linear in y i . However, if we insert y i = g j w ij x j into the learning rule, the learning dynamics become nonlinear in x j . Nonlinearity implies that we cannot define a timescale of plasticity, unless we find a fixed point where dw ij /dt vanishes.
Are there fixed points of the dynamics? There is a fixed point if the postsynaptic or the presynaptic rate is zero. However, if the neuron is embedded in a network, it is reasonable to assume that several presynaptic neurons including the presynaptic neuron j are active. Unless all weights w ij are zero, the postsynaptic neuron is therefore also active, and the weight w ij increases.
The argument can be formalized to show that w ij = 0 is an unstable fixed point. If we increase w ij by just a little, the output y i also increases which increases w ij even further, which closes the positive feedback loop. More generally, models of Hebbian plasticity that are useful for memory formation all have an unstable fixed point. One important role of Rapid Compensatory Processes (RCPs) in network models with Hebbian plasticity is therefore to create (additional) stable fixed points for the learning dynamics as we will see later on in this section.
The simple example above only has a trivial fixed point at zero activity (zero weights). Moreover, it is lacking the notion of long-term depression (LTD). Plausible plasticity models have additional stationary points defined by the plasticity threshold between long-term potentiation (LTP) and LTD (Fig. 2a). Inspired by experimental data [1][2][3], the transition from LTP to LTD depends in S3 models on the state of the postsynaptic neuron, e.g., its membrane potential, calcium level, inter spike interval [4][5][6][7][8]. As a paradigmatic example, which stands for a plethora of different plasticity rules with a postsynaptic threshold, we consider the following rate-based nonlinear Hebbian rule with a positive constant η. Whenever the postsynaptic firing rate y i equals the value of θ, the weight w ij does not change. Thus this learning rule has a fixed point at the threshold y i = θ. For y i larger than θ the synaptic weights increase which corresponds to the induction of LTP in the model. For y i smaller than θ the synaptic weights decrease and LTD is induced.
Let us now embed this learning rule in a network. For the sake of simplicity we assume the gain function to be linear, g j w ij x j = j w ij x j . Moreover, we assume that (i) all weights w ij have the same value w ij = w and (ii) all N neurons in the network fire with the same firing rate x j = 1/N , so that y = w. Inserting these assumptions into Eq. (S4) yields which characterizes the weight change induced by synaptic plasticity. We now linearize Expression (S5) at the stationary point w = θ: This is a linear differential equation with a solution w(t) that, for w(t) > θ, explodes exponentially fast on the timescale τ = 1 ηθ (cf. discussion of Eq. (S1)). Thus, in this example, w = θ is an unstable fixed point and the linearization procedure has enabled us to identify the timescale of plasticity induction. In practical implementations of a plasticity model, the exponential growth of the synaptic weights w ij would stop when they obtain their maximal value w max . However, if all synapses onto a postsynaptic neuron, or even all synapses in a neural network sit at their upper bound, the network cannot function as a memory.
Note that the occurrence of the parameter θ in the timescale τ is a hallmark of the nonlinearity of the full system. Just like in the well-known Hodgkin-Huxley model where the timescale of the activation and inactivation variable depends on the voltage, the effective timescale τ of a learning rule will depend on multiple factors such as the presynaptic activity, the slope of the gain function, or the threshold θ between LTD and LTP.
Even though τ in the example above is not the same as the induction timescale of long-term plasticity, in experiments in which only a single presynaptic pathway is stimulated, the two are related. With our formalism we can also account for the strength of the recurrent feedback that is received by a neuron embedded into a network. Just repeat the above analysis under the assumption S4 that the presynaptic and postsynaptic neurons are mutually connected, of the same type, and fire at the same rate [9]. Whatever you consider as a reasonable scenario, the timescale τ characterizes how quickly a neuron or an entire recurrent network can generate positive feedback and is able to "run away".
Instead of writing down a differential equation for the weights w, as in Eq. (S5), we could have written the system in terms of y, too. A formulation in terms of the firing rates y highlights the fact that we have to think about run-away effects of synapses as being linked to run-away effects of neuronal activity. Note further, that in realistic neuronal networks the presynaptic activity fluctuates and is different between one neuron and the next. Fluctuations give rise to a covariance matrix C = 1 n n t=1 x i (t)x j (t) which may look complicated at a first glance. However, due to the symmetry of C, there always exists a basis in which C is diagonal. When working in this basis, the plasticity equations decouple and take the shape of Eq. (S5) with different values of η.
This analytical formalism is quite general [9] (see Yger and Gilson [10] for a review) and can be applied to spike-timing-dependent plasticity (STDP), spiking neurons and spiking neural networks.
However, because such systems are often stochastic and usually high-dimensional it can become intractable to explicitly compute the state of each neuron x i and each synapse w ij . However, it is often possible to consider instead the average change across all synapses dw ij dt and activities dy i dt respectively. Under fairly general conditions these averages take the functional form of the simple rate models above which are low-dimensional and analytically tractable. Despite this enormous dimensionality reduction from a large spiking neural network with a quantitative STDP model, these mean field models give surprisingly accurate quantitative predictions about bifurcation parameters such as τ crit ) at which instability occurs [9].