Spike response model explained

The spike response model (SRM)^[1] is a spiking neuron model in which spikes are generated by either a deterministic^[2] or a stochastic threshold process. In the SRM, the membrane voltage is described as a linear sum of the postsynaptic potentials (PSPs) caused by spike arrivals to which the effects of refractoriness and adaptation are added. The threshold is either fixed or dynamic. In the latter case it increases after each spike. The SRM is flexible enough to account for a variety of neuronal firing pattern in response to step current input. The SRM has also been used in the theory of computation to quantify the capacity of spiking neural networks; and in the neurosciences to predict the subthreshold voltage and the firing times of cortical neurons during stimulation with a time-dependent current stimulation.^[3] The name Spike Response Model points to the property that the two important filters

\varepsilon and η of the model can be interpreted as the response of the membrane potential to an incoming spike (response kernel \varepsilon , the PSP) and to an outgoing spike (response kernel η , also called refractory kernel). The SRM has been formulated in continuous time and in discrete time.^[4] The SRM can be viewed as a generalized linear model (GLM)^{[5] [6]} or as an (integrated version of) a generalized integrate-and-fire model with adaptation.^{[7] [8]}

Model equations for SRM in continuous time

In the SRM, at each moment in time t, a spike can be generated stochastically with instantaneous stochastic intensity or 'escape function'

\rho(t) = f(V(t) - \vartheta(t))

that depends on the momentary difference between the membrane voltage and the dynamic threshold

\vartheta(t) .

The membrane voltage at time t is given by

V(t) = \sum _{f} η (t - t ^{f)} + \int ^{inf ty} _{0} \kappa(s) I(t - s) ds + V _{rest}

where is the firing time of spike number f of the neuron, is the resting voltage in the absence of input, is the input current at time t − s and

\kappa(s) is a linear filter (also called kernel) that describes the contribution of an input current pulse at time t − s to the voltage at time t. The contributions to the voltage caused by a spike at time t ^{f} are described by the refractory kernel η (t - t ^{f)} . In particular,η (t - t ^{f)} describes the time course of the action potential starting at time t ^{f} as well as the spike-afterpotential.

The dynamic threshold

\vartheta(t) is given by\vartheta(t) = \vartheta _{0} + \sum _{f} \theta ^{f} _{1 (t - t} )

where

\vartheta _{0} is the firing threshold of an inactive neuron and \theta ^{f)} _{1 (t - t} describes the increase of the threshold after a spike at time t ^{f} . In case of a fixed threshold [i.e., <math>\theta_1(t-t^f)</math>=0], the refractory kernel η (t - t ^{f)} should include only the spike-afterpotential, but not the shape of the spike itself.

A common choice for the 'escape rate'

f (that is consistent with biological data) isf(V - \vartheta) = (1)/(\tau _{0}) \exp[\beta(V - \vartheta)]

where

\tau _{0} is a time constant that describes how quickly a spike is fired once the membrane potential reaches the threshold and \beta is a sharpness parameter. For \beta\to inf ty the threshold becomes sharp and spike firing occurs deterministically at the moment when the membrane potential hits the threshold from below. The sharpness value found in experiments is 1 /\beta ≈ 4 mV which that neuronal firing becomes non-neglibable as soon the membrane potential is a few mV below the formal firing threshold. The escape rate process via a soft threshold is reviewed in Chapter 9 of the textbook Neuronal Dynamics.

In a network of N SRM neurons

1 \le i \le N , the membrane voltage of neuron i is given byV _{i(t)=} \sum _{f} η _{i(t-t} ^{f} ^{i} ) + \sum ^{N} _{j =1} w _{ij} \sum _{f'} \varepsilon _{ij} (t - t ^{f'} _{j} ) + V _{rest}

where

t ^{f'} _{j} are the firing times of neuron j (i.e., its spike train), and η ^{f} _{i)} describes the time course of the spike and the spike after-potential for neuron i, w _{ij} and \varepsilon _{ij} (t - t ^{f'} _{j} ) describe the amplitude and time course of an excitatory or inhibitory postsynaptic potential (PSP) caused by the spike t ^{f'} _{j} of the presynaptic neuron j. The time course \varepsilon _{ij} (s) of the PSP results from the convolution of the postsynaptic current I(t) caused by the arrival of a presynaptic spike from neuron j.

Model equations for SRM in discrete time

For simulations, the SRM is usually implemented in discrete time.^[9] In time step

t _{n} of duration \Delta t , a spike is generated with probabilityP _{F(t} _{n)} = F(V(t _{n)-\vartheta(t} _{n))}

that depends on the momentary difference between the membrane voltage and the dynamic threshold

\vartheta . The function F is often taken as a standard sigmoidal F(x) = 0 . 5 [ 1 + \tanh(\gamma x)] ^[10] with steepness parameter \gamma . But the functional form of F can also be calculated from the stochastic intensity f in continuous time as F(y _{n) ≈} 1 - \exp[y _{n} \Delta t] where y _{n} = V(t _{n)-\vartheta(t} _{n)} is the distance to threshold.

The membrane voltage

V(t _{n)} in discrete time is given byV(t _{n)} = \sum _{f} η (t ^{f} _{n - t} ) + \sum ^{inf ty} _{m =1} \kappa(m\Delta t) I(t _{n-m\Delta} t) + V _{rest}

where is the discretized firing time of the neuron, is the resting voltage in the absence of input, and

I(t _{k)} is the input current at time t _{k} (integrated over one time step). The input filter \kappa(s) and the spike-afterpotential η (s) are defined as in the case of the SRM in continuous time.

For networks of SRM neurons in discrete time we define the spike train of neuron j as a sequence of zeros and ones,

\{X _{j(t} _{m)\in} \{ 0 , 1 \}; m =1 , 2 , 3 , ... \} and rewrite the membrane potential asV _{i(t} _{n)} = \sum _{mη} _{i(t} _{n-t} _{m)} X _{i(t} _{m)} + \sum _{j} w _{ij} \sum _{m} \varepsilon _{ij} (t _{n-t} _{m)} X _{j(t} _{m)} + V _{rest}

In this notation, the refractory kernel

\kappa(s) and the PSP shape \varepsilon _{ij} (s) can be interpreted as linear response filters applied to the binary spike trains X _{j} .

Main applications of the SRM

Theory of computation with pulsed neural networks

Since the formulation as SRM provides an explicit expression for the membrane voltage (without the detour via a differential equations), SRMs have been the dominant mathematical model in a formal theory of computation with spiking neurons.^{[11] [12] [13]}

Prediction of voltage and spike times of cortical neurons

The SRM with dynamic threshold has been used to predict the firing time of cortical neurons with a precision of a few milliseconds. Neurons were stimulated, via current injection, with time-dependent currents of different means and variance while the membrane voltage was recorded. The reliability of predicted spikes was close to the intrinsic reliability when the same time-dependent current was repeated several times. Moreover, extracting the shape of the filters

\kappa(s) and η (s) directly from the experimental data revealed that adaptation extends over time scales from tens of milliseconds to tens of seconds.^{[14] [15]} Thanks to the convexity properties of the likelihood in Generalized Linear Models, parameter extraction is efficient.^[16]

Associative memory in networks of spiking neurons

SRM0 neurons have been used to construct an associative memory in a network of spiking neurons. The SRM network which stored a finite number of stationary patterns as attractors using a Hopfield-type connectivity matrix^[17] was one of the first examples of attractor networks with spiking neurons.^{[18] [19] [20]}

Population activity equations in large networks of spiking neurons

For SRM neurons, an important variable characterizing the internal state of the neuron is the time since the last spike (or 'age' of the neuron) which enters into the refractory kernel

η (s) . The population activity equations for SRM neurons can be formulated alternatively either as integral equations,^[21] or as partial differential equations for the 'refractory density'. Because the refractory kernel may include a time scale slower than that of the membrane potential, the population equations for SRM neurons provide powerful alternatives^{[22] [23]} to the more broadly used partial differential equations for the 'membrane potential density'.^{[24] [25]} Reviews of the population activity equation based on refractory densities can be found in as well in Chapter 14 of the textbook Neuronal Dynamics.

Spike patterns and temporal code

SRMs are useful to understand theories of neural coding. A network SRM neurons has stored attractors that form reliable spatio-temporal spike patterns (also known as synfire chains^[26]) example of temporal coding for stationary inputs. Moreover, the population activity equations for SRM exhibit temporally precise transients after a stimulus switch, indicating reliable spike firing.

4. History and relation to other models

The Spike Response Model has been introduced in a series of papers between 1991 and 2000.^[27] The name Spike Response Model probably appeared for the first time in 1993. Some papers used exclusively the deterministic limit with a hard threshold others the soft threshold with escape noise. Precursors of the Spike Response Model are the integrate-and-fire model introduced by Lapicque in 1907 as well as models used in auditory neuroscience.^[28]

SRM0

An important variant of the model is SRM0 which is related to time-dependent nonlinear renewal theory. The main difference to the voltage equation of the SRM introduced above is that in the term containing the refractory kernel

η (s) there is no summation sign over past spikes: only the most recent spike matters. The model SRM0 is closely related to the inhomogeneous Markov interval process^[29] and to age-dependent models of refractoriness.^{[30] [31] [32]}

GLM

The equations of the SRM as introduced above are equivalent to Generalized Linear Models in neuroscience (GLM). In the neuroscience, GLMs have been introduced as an extension of the Linear-Nonlinear-Poisson model (LNP) by adding self-interaction of an output spike with the internal state of the neuron (therefore also called 'Recursive LNP'). The self-interaction is equivalent to the kernel

η (s) of the SRM. The GLM framework enables to formulate a maximum likelihood approach^[33] applied to the likelihood of an observed spike train under the assumption that an SRM could have generated the spike train. Despite the mathematical equivalence there is a conceptual difference in interpretation: in the SRM the variable V is interpreted as membrane voltage whereas in the recursive LNP it is a 'hidden' variable to which no meaning is assigned. The SRM interpretation is useful if measurements of subthreshold voltage are available whereas the recursive LNP is useful in systems neuroscience where spikes (in response to sensory stimulation) are recorded extracellulary without access to the subthreshold voltage.

Adaptive leaky integrate-and-fire models

A leaky integrate-and-fire neuron with spike-triggered adaptation has a subthreshold membrane potential generated by the following differential equations

\tau _{m} (d V (t))/(d t) = R I(t) - [V (t) - E _{rest} ] - R \sum _{k} w _{k} \tau _{k} (d w _{k} (t))/(d t) = - w _{k} + b _{k} \tau _{k} \sum _{f} \delta (t - t ^{f)}

where

\tau _{m} is the membrane time constant and is an adaptation current number, with index k, is the resting potential and is the firing time of the neuron and the Greek delta denotes the Dirac delta function. Whenever the voltage reaches the firing threshold the voltage is reset to a value below the firing threshold. Integration of the linear differential equations gives a formula identical to the voltage equation of the SRM. However, in this case, the refractory kernel η (s) does not include the spike shape but only the spike-afterpotential. In the absence of adaptation currents, we retrieve the standard LIF model which is equivalent to a refractory kernel η (s) that decays exponentially with the membrane time constant \tau _{m} .

External links

• Spike Response Model, Chapter 6.4 of the textbook Neuronal Dynamics
• 'soft threshold' and escape noise, Chapter 9 of the textbook Neuronal Dynamics
• Quasi-Renewal Theory Chapter 14 of the textbook Neuronal Dynamics.
• Spike Response Model, from Scholarpedia^[34]

Notes and References

1. Gerstner. Wulfram. Ritz. Raphael. van Hemmen. J. Leo. 1993-10-01. Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biological Cybernetics. en. 69. 5. 503–515. 10.1007/BF00199450. 7903867. 6195748. 1432-0770.
2. Gerstner. Wulfram. van Hemmen. J. Leo. Cowan. Jack D.. 1996-11-15. What Matters in Neuronal Locking?. Neural Computation. 8. 8. 1653–1676. 10.1162/neco.1996.8.8.1653. 8888612. 1301248. 0899-7667.
3. Jolivet. Renaud. Rauch. Alexander. Lüscher. Hans-Rudolf. Gerstner. Wulfram. August 2006. Predicting spike timing of neocortical pyramidal neurons by simple threshold models. Journal of Computational Neuroscience. en. 21. 1. 35–49. 10.1007/s10827-006-7074-5. 16633938. 8911457. 0929-5313.
4. Gerstner. Wulfram. Hemmen. J. Leo van. 1992-01-01. Associative memory in a network of 'spiking' neurons. Network: Computation in Neural Systems. 3. 2. 139–164. 10.1088/0954-898X_3_2_004. 0954-898X.
5. Truccolo. Wilson. Eden. Uri T.. Fellows. Matthew R.. Donoghue. John P.. Brown. Emery N.. 2005-02-01. A Point Process Framework for Relating Neural Spiking Activity to Spiking History, Neural Ensemble, and Extrinsic Covariate Effects. Journal of Neurophysiology. 93. 2. 1074–1089. 10.1152/jn.00697.2004. 15356183. 0022-3077.
6. Pillow. Jonathan W.. Shlens. Jonathon. Paninski. Liam. Sher. Alexander. Litke. Alan M.. Chichilnisky. E. J.. Simoncelli. Eero P.. August 2008. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature. en. 454. 7207. 995–999. 10.1038/nature07140. 1476-4687. 2684455. 18650810. 2008Natur.454..995P.
7. Book: Gerstner, Wulfram.. Neuronal dynamics : from single neurons to networks and models of cognition. Kistler, Werner M., 1969-, Naud, Richard., Paninski, Liam.. 24 July 2014. 978-1-107-06083-8. Cambridge, United Kingdom. 861774542.
8. Book: Gerstner, Wulfram.. Spiking neuron models : single neurons, populations, plasticity. 2002. Cambridge University Press. Kistler, Werner M., 1969-. 0-511-07817-X. Cambridge, U.K.. 57417395.
9. Gerstner. Wulfram. 2000-01-01. Population Dynamics of Spiking Neurons: Fast Transients, Asynchronous States, and Locking. Neural Computation. 12. 1. 43–89. 10.1162/089976600300015899. 10636933. 7832768. 0899-7667.
10. GERSTNER. Wulfram. 1991. [papers.nips.cc/paper/371-associative-memory-in-a-network-of-biological-neurons.pdf Associative memory in a network of biological neurons]. Advances in Neural Information Processing Systems. 3. 84–90.
11. Book: Pulsed neural networks. 1999. MIT Press. Maass, Wolfgang, 1949 August 21-, Bishop, Christopher M.. 978-0-262-27876-8. Cambridge, Mass.. 42856203.
12. Bohte. Sander M.. Kok. Joost N.. La Poutré. Han. 2002-10-01. Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing. en. 48. 1. 17–37. 10.1016/S0925-2312(01)00658-0. 0925-2312.
13. Maass. Wolfgang. 1997-02-01. Fast Sigmoidal Networks via Spiking Neurons. Neural Computation. 9. 2. 279–304. 10.1162/neco.1997.9.2.279. 9117904. 34984298 . 0899-7667.
14. Pozzorini. Christian. Naud. Richard. Mensi. Skander. Gerstner. Wulfram. July 2013. Temporal whitening by power-law adaptation in neocortical neurons. Nature Neuroscience. en. 16. 7. 942–948. 10.1038/nn.3431. 23749146. 1873019. 1546-1726.
15. Mensi. Skander. Naud. Richard. Pozzorini. Christian. Avermann. Michael. Petersen. Carl C. H.. Gerstner. Wulfram. 2011-12-07. Parameter extraction and classification of three cortical neuron types reveals two distinct adaptation mechanisms. Journal of Neurophysiology. 107. 6. 1756–1775. 10.1152/jn.00408.2011. 22157113. 0022-3077.
16. Pozzorini. Christian. Mensi. Skander. Hagens. Olivier. Naud. Richard. Koch. Christof. Gerstner. Wulfram. 2015-06-17. Automated High-Throughput Characterization of Single Neurons by Means of Simplified Spiking Models. PLOS Computational Biology. en. 11. 6. e1004275. 10.1371/journal.pcbi.1004275. 1553-7358. 4470831. 26083597. 2015PLSCB..11E4275P . free .
17. Hopfield. J. J.. 1982-04-01. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences. en. 79. 8. 2554–2558. 10.1073/pnas.79.8.2554. 0027-8424. 346238. 6953413. 1982PNAS...79.2554H. free .
18. Amit. Daniel J.. Tsodyks. M. V.. 1991-01-01. Quantitative study of attractor neural network retrieving at low spike rates: I. substrate—spikes, rates and neuronal gain. Network: Computation in Neural Systems. 2. 3. 259–273. 10.1088/0954-898X_2_3_003. 0954-898X.
19. Amit. D.. 1997-04-01. Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral Cortex. 7. 3. 237–252. 10.1093/cercor/7.3.237. 9143444. 1460-2199. free.
20. Brader. Joseph M.. Senn. Walter. Fusi. Stefano. 2007-09-20. Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics. Neural Computation. 19. 11. 2881–2912. 10.1162/neco.2007.19.11.2881. 17883345. 5933554. 0899-7667.
21. Schwalger. Tilo. Deger. Moritz. Gerstner. Wulfram. 2017-04-19. Graham. Lyle J.. Towards a theory of cortical columns: From spiking neurons to interacting neural populations of finite size. PLOS Computational Biology. en. 13. 4. e1005507. 10.1371/journal.pcbi.1005507. 1553-7358. 5415267. 28422957. 1611.00294. 2017PLSCB..13E5507S . free .
22. Naud. Richard. Gerstner. Wulfram. 2012-10-04. Coding and Decoding with Adapting Neurons: A Population Approach to the Peri-Stimulus Time Histogram. PLOS Computational Biology. en. 8. 10. e1002711. 10.1371/journal.pcbi.1002711. 1553-7358. 3464223. 23055914. 2012PLSCB...8E2711N . free .
23. Schwalger. Tilo. Chizhov. Anton V. 2019-10-01. Mind the last spike — firing rate models for mesoscopic populations of spiking neurons. Current Opinion in Neurobiology. Computational Neuroscience. en. 58. 155–166. 10.1016/j.conb.2019.08.003. 31590003. 1909.10007. 202719555. 0959-4388.
24. Treves. Alessandro. January 1993. Mean-field analysis of neuronal spike dynamics. Network: Computation in Neural Systems. en. 4. 3. 259–284. 10.1088/0954-898X_4_3_002. 0954-898X.
25. Brunel. Nicolas. 2000-05-01. Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons. Journal of Computational Neuroscience. en. 8. 3. 183–208. 10.1023/A:1008925309027. 10809012. 1849650. 1573-6873.
26. Abeles. Moshe. 2009. Synfire chains. Scholarpedia. en. 4. 7. 1441. 10.4249/scholarpedia.1441. 2009SchpJ...4.1441A. 1941-6016. free.
27. Gerstner. Wulfram. 1995-01-01. Time structure of the activity in neural network models. Physical Review E. 51. 1. 738–758. 10.1103/PhysRevE.51.738. 9962697. 1995PhRvE..51..738G.
28. Weiss. Thomas F.. 1966-11-01. A model of the peripheral auditory system. Kybernetik. en. 3. 4. 153–175. 10.1007/BF00290252. 5982096. 30861035 . 1432-0770.
29. Kass. Robert E.. Ventura. Valérie. 2001-08-01. A Spike-Train Probability Model. Neural Computation. en. 13. 8. 1713–1720. 10.1162/08997660152469314. 11506667. 9909632 . 0899-7667.
30. Johnson. Don H.. Swami. Ananthram. 1983-08-01. The transmission of signals by auditory-nerve fiber discharge patterns. The Journal of the Acoustical Society of America. 74. 2. 493–501. 10.1121/1.389815. 6311884. 1983ASAJ...74..493J . 0001-4966.
31. Berry. Michael J.. Meister. Markus. 1998-03-15. Refractoriness and Neural Precision. The Journal of Neuroscience. en. 18. 6. 2200–2211. 10.1523/JNEUROSCI.18-06-02200.1998. 0270-6474. 6792934. 9482804.
32. Gaumond. R P. Molnar. C E. Kim. D O. September 1982. Stimulus and recovery dependence of cat cochlear nerve fiber spike discharge probability.. Journal of Neurophysiology. en. 48. 3. 856–873. 10.1152/jn.1982.48.3.856. 6290620. 0022-3077.
33. Brillinger. D. R.. 1988-08-01. Maximum likelihood analysis of spike trains of interacting nerve cells. Biological Cybernetics. en. 59. 3. 189–200. 10.1007/BF00318010. 3179344. 10164135. 1432-0770.
34. Gerstner. Wulfram. 2008. Spike-response model. Scholarpedia. en. 3. 12. 1343. 10.4249/scholarpedia.1343. 2008SchpJ...3.1343G . 1941-6016. free.