Introduction to Neural Computation – 9.
40
• Prof. Michale Fee, Instructor
• Daniel Zysman, Technical instructor
1
Texts: Selected readings
• Berg, Random Walks in Biology
• Dayan & Abbott, Theoretical Neuroscience.
• Hille, Ionic Channels of Excitable Membranes
…and others
2
What is neural computation?
• Brain and cognitive sciences are no longer primarily descriptive
– Engineering-level descriptions of brain systems.
Diagram Ⓒ Jeff Dean (adapted from DiCarlo & Cox, 2007). All rights
reserved. This content is excluded from our Creative Commons License.
For more information see https://ocw.mit.edu/help/faq-fair-use/.
Figure courtesy of Wormbook. License: CC BY. Eisenmann, D. M., ed.
The C. elegans Research Community, doi/10.1895/wormbook.1.7.1. 3
New technologies for neuronal activity
measurements
Video
YaleCampus. "Imaging Brain Activity
Across the Mouse Cortex." YouTube.
Crair Lab, Yale Univ
4
What is neural computation?
• Brain and cognitive sciences are no longer primarily descriptive
– Engineering-level descriptions of brain systems.
• Use mathematical techniques to analyze neural data in a way that
allows us to relate it to mathematical models.
• In this course we will have the added component that we will apply
these techniques to understand the circuits and computational
principles that underlie animal behavior.
5
Neural circuits that control bird song
See Lecture 1 video recording for playback
6
What is neural computation?
• Computational and quantitative approaches are also important in
cognitive science.
• Importance of computation and quantitation in medical sciences
7
Course Goals
• Understand the basic biophysics of neurons and networks
and other principles underlying brain and cognitive
functions
• Use mathematical techniques to
– analyze simple models of neurons and networks
– do data analysis of behavioral and neuronal data (compact
representation of data)
• Become proficient at using numerical methods to
implement these techniques (MATLAB®)
8
Topics
Neuronal biophysics and model neurons Differential equations
Neuronal responses and tuning curves Spike sorting, PSTHs and firing rates
Neural coding and receptive fields Correlation and convolution
Feed forward networks and perceptrons Linear algebra
Data analysis, dimensionality reduction Principle Component Analysis and SVD
Short-term memory, decision making Recurrent networks, eigenvalues
Sensory integration Bayes rule
9
Skills you will have
• Translate a simple model of neurons and neural circuits into a
mathematical model
• Be able to simulate simple models using MATLAB®
• Be able to analyze neuronal data (or model output) using
MATLAB®
• Be able to visualize high dimensional data.
• Be able to productively contribute to research in a
neuroscience lab!
10
Problem sets
• MATLAB® will be used extensively for the problem sets.
– Free for students. Please install on your laptop.
• We will use live scripts for Pset submissions.
11
Introduction to Neural
Computation
Michale Fee
MIT BCS 9.40 — 2018
Lecture 1 – Ionic Currents
12
A mathematical model of a neuron
• Equivalent circuit model
gNa gK gL
Vm
C
EK EL Ie
+ ENa
+ +
100
• A conceptual model based on simple
Vm
components from electrical circuits
0
−100
0 100 200 300
Time (ms)
• A mathematical model that we can use
to calculate properties of neurons 0.2
0.1
0
0 100 200 300
Time (ms)
13
Why build a model of a neuron?
• Neurons are very complex.
• Different neuron types are defined by the genes that are expressed and their
complement of ion channels
• Ion channels have dynamics at different timescales, voltage ranges, inactivation
Figures removed due to copyright restrictions. Left side is Figure 3a: Spectral tSNE plot of 13,079
neurons, colored according to the results of iterative subclustering. Campbell, J., et al. "A
molecular census of arcuate hypothalamus and median eminence cell types." Nature Neuroscience
20, pages 484–496 (2017). Right side is Figure 1: Representation of the amino acid sequence
relations of the minimal pore regions of the voltage-gated ion channel superfamily. Yu, F.H. and
W.A. Catterall. "The VGL-Chanome: A Protein Superfamily Specialized for Electrical Signaling and
Ionic Homeostasis." Science's STKE05 Oct 2004: re15.
14
Neurons are extremely complex
• Ion channel and morphological diversity lead to diversity
of firing patterns
• It’s hard to guess how morphology and ion channels lead
to firing patterns
• … and how firing patterns control circuit behavior
Figures removed due to copyright restrictions. Left side source unknown. Right side is Figure 6.1:
Multiple firing patterns in cortical neurons. In: Gerstner, W., et al. Neuronal Dynamics. Cambridge
University Press.
15
A mathematical model of a neuron
• Equivalent circuit model
gNa gK gL
Vm
C
EK EL Ie
+ ENa
+ +
• Different parts of this circuit do different interesting things
– Power supplies
– Integrator of past inputs
– Temporal filter to smooth inputs in time
– Spike generator
– Oscillator 16
Ionic currents
gNa gK gL
Vm
C
EK EL Ie
+ ENa
+ +
What are the wires of the brain?
In the brain (in neurons), current flow results from the movement of
ions in aqueous solution (water).
17
Basic electrochemistry
• Water is a polar solvent
• Intracellular and extracellular space is filled with salt
solution (~100mM) H+
H+ H+
– 6x1019 ions per cm3 (25Å spacing) O- O- H+ O-
Na+
H+ O- O- H+ Cl- H+
H+
H+ H+
H+ H+ O-
• Currents flow through a salt solution by two key
mechanisms: + - ΔV
o Diffusion
I
o Drift in an electric field
18
Learning objectives for Lecture 1
• To understand how the timescale of diffusion relates to
length scales
• To understand how concentration gradients lead to
currents (Fick’s First Law)
• To understand how charge drift in an electric field leads
to currents (Ohm’s Law and resistivity)
19
Thermal energy
• Every degree of freedom comes to thermal equilibrium with
an energy proportional to temperature (Kelvin, K)
• The proportionality constant is the Boltzmann constant (k)
kT = 4x10 −21 Joules at 300K)
• Kinetic energy : 1 2 1 kT
mvx = kT vx2 =
2 2 m
• The mass of a sodium ion is 3.8x10-26 kg
vx2 = 10 5 m 2 / s 2 ⇒ vx = 3.2×10 2 m/s
This would cross this 10m classroom in 3/10 second!
Here we follow ‘Random Walks in Biology’
Howard C. Berg, Princeton Univ Press 1993 20
What is diffusion?
• A particle in solution undergoes collisions with water molecules very
often (~1013 times per second!) that constantly change its direction
of motion.
Collisions produce a ‘random walk’ in space
21
Spatial and temporal scales
Diffusion is fast at short length scales and slow at long length scales.
• To diffuse across a cell body (10um) it takes an ion 50ms
• To diffuse down a dendrite (1mm) it takes about 10min
• How long does it take an ion to diffuse down a motor neuron axon (1m)?
10 years!
22
Distribution of particles resulting from diffusion
in 1-D
• On average particles stay
clustered around initial position
• Particles spread out around
initial position
• We can compute analytically
properties of this distribution!
23
• An ensemble of particles diffusing
from a point acquires a Gaussian
distribution
• This arises from a binomial
distribution for large number of
Gaussian Distribution
time-steps (The probability of the
particle moving exactly k steps to
the right in n steps will be:
⎛ n⎞ k
P(k;n, p) = ⎜ ⎟ p (1− p)n−k
⎝ k⎠
0
1 − x 2 /4 Dt
lim P(k;n, p) = e
np→∞ 4π Dt
24
Random walk in one dimension
• We can mathematically analyze the properties of an ensemble of
particles undergoing a random walk
• Consider a particle moving left or right at a fixed velocity vx for a τ
time before a collision.
• Imagine that each collision randomly resets the direction
• Thus, on every time-step,
– half the particles step right by a distanceδ = +vx τ
– and half the particles step to the left by a distance δ
25
Random Walk in 1-D
• Assume that we have N particles that start at position x=0 at time
t=0
• xi(n) = the position of the ith particle on time-step n: n = t / τ
• Assume the movement of each particle is independent
• Thus, we can write the position of each particle at time-step n as a
function of the position at previous time-step
xi (n) = xi (n − 1) ± δ
• Use this to compute how the distribution evolves in time!
26
Average displacement is zero
• What is the average position of our ensemble?
1
xi (n) i = ∑
N i
xi (n)
xi (n) = xi (n − 1) ± δ
1
=
N
∑ [ x (n − 1) ± δ ]
i
i
0
1 1
=
N
∑ [ x (n − 1)]
i + ∑
N i
( ±δ )
i
xi (n) i = xi (n − 1) i
27
Here we follow ‘Random Walks in Biology’ Howard C. Berg, Princeton Univ Press 1993
Distribution of particles resulting from diffusion
in 1-D
• On average particles stay
clustered around initial
position
• Particles spread out around
initial position
• We can compute analytically
properties of this
distribution!
28
How far does a particle travel due to diffusion?
• We want to compute an average ‘absolute value’ distance from
origin… Root mean square distance
x(n) → x 2 (n)
xi (n) = xi (n − 1) ± δ
Compute variance
xi2 (n) = (xi (n − 1) ± δ )2
x 2 (n) = 1
N ∑ i (n)
x 2
= xi2 (n − 1) ± 2δ xi (n − 1) + δ 2
i
0
x 2 (n) = x 2 (n − 1) + ±2δ xi (n − 1) + δ 2
x 2 (n) = x 2 (n − 1) + δ 2
29
How far does a particle travel due to diffusion?
x 2 (n) = x 2 (n − 1) + δ 2
• Note that at each time-step, the variance grows by δ 2
x 2 (0) = 0 , x 2 (1) = δ 2 , x 2 (2) = 2δ 2 , ... x 2 (n) = nδ 2
δ 2
t
xi 2 ( t ) = , n = t /τ
τ
xi 2 = 2Dt, D = δ 2 / 2τ (Diffusion coefficient)
x 2 = 2Dt
30
Spatial and temporal scales
L = 2Dt L2 = 2Dt t = L2 2D
Diffusion is fast at short length scales and slow at long length scales.
Typical diffusion constants for small molecules and ions are ~10-5
cm2/s
• L = 10μm = 10-3 cm t = 10-6(cm2)/2x10-5(cm2/s) = 50 ms
• L = 1mm = 10-1 cm t = 10-2(cm2)/2x10-5(cm2/s) = 500 s
• L = 1000mm = 102 cm t = 104(cm2)/2x10-5(cm2/s) =
500,000,000 seconds!!
31
Fick’s first law
• Diffusion produces a net flow of particles from regions of high concentration to
regions of lower concentration.
• The flux of particles is proportional to the concentration gradient.
N(x) is the number of particles in
the box at position x
1 N(x)
J x = −D [ϕ(x + δ)−ϕ(x)] N(x+δ)
δ
x x+δ
∂ϕ
J x = −D 1 N(x)
∂x 1 N(x + δ)
2
2
1
2 [ N(x)− N(x + δ)]
is the net number of particles moving to
the right in an interval of time τ 32
Diffusion produces a net flux of particles down a
gradient
K+ K+
K+ K+ K+
K+ K+
K+ K+ K+
K+ K+
K+ K+
• Each particle diffuses
independently and ϕ(x)
randomly! ∂ϕ
J x = −D
∂x
• And yet concentration
gradients produce currents! x
• Eventually all concentration gradients go away… 33
Current flow in neurons obeys Ohm’s Law
In a wire, current flow is
proportional to voltage difference
I
+
ΔV
Ohm’s Law ΔV R
I=
R
where
– I is current (Amperes, A)
– ΔV is voltage (Volts, V)
– R is resistance (Ohms, Ω)
34
Where does Ohm’s Law come from?
Consider a beaker filled with salt solution, two electrodes, and a
battery that produces a voltage difference between the electrodes.
• The electric field produces a force which, in a solution, causes an
ion to drift with a constant velocity — a current
- + ΔV V- V+
K+
I ! E=
ΔV
E L
L V(x)
! ! ΔV
• Force: F=qE 35
x
Ion currents in an electric field
Currents are also caused by the drift of ions in the presence of an
electric field.
• The electric field produces a force which, in a solution, causes an
ion to drift with a constant velocity — a current
- + ΔV V- V+
K+
I ! E=
ΔV
E L
V(x)
L
ΔV
• Why constant velocity? 36
x
Ion currents in an electric field
Currents are also caused by the drift of ions in the presence of an
electric field.
• Einstein realized that this is just a result of viscous drag (or friction)
! !
F = f vd
V- V+
• Einstein –Smoluchovski relation K+
f = kT / D
! E=
ΔV
E L
• Drift velocity is given by
V(x)
! D ! D !
vd =
kT
F =
kT
( qE ) ΔV
37
x
Ion currents in an electric field
Currents are also caused by the drift of ions in the presence of an
electric field.
• The electric field produces a force which, in a solution, causes an
ion to drift with a constant velocity — a current
V- V+
I ∝ vd A K+
! E=
ΔV
E L
ΔV
I ∝EA= A V(x)
L
ΔV
38
x
Ohm’s Law in solution
In a solution, current flow per unit area is proportional to
voltage gradients
⎛ 1 ⎞ ΔV ρ = resistivity (Ω.m) 1
I= ⎜ ⎟ A I= ΔV
⎝ρ⎠ L R
• Let’s make this look more like Ohm’s Law
⎛ A⎞ - + ΔV
I = ⎜ ⎟ ΔV
⎝ ρL ⎠ Surface area A
• Thus the resistance is given by: I
ρL
R=
A L
39
Resistivity of intra/extra cellular space
- + V
Surface
• Resistance of a volume of area A
conductive medium is given by
ρL I
R=
A
L
• ρ = 1.6 μΩ.cm for copper
• ρ = ~60 Ω.cm for mammalian saline – the brain
has lousy conductors!
• The brain has many specializations to deal with lousy wires…
40
Learning objectives for Lecture 1
• To understand how the timescale of diffusion relates to
length scales
– Distance diffused grows as the square root of time
• To understand how concentration gradients lead to
currents (Fick’s First Law)
– Concentration differences lead to particle flux, proportional to
gradient
• To understand how charge drift in an electric field leads
to currents (Ohm’s Law and resistivity)
41
(Extra slide) Derivation of resistivity
Current density (Coulombs per second per unit area) is just drift
velocity times the density of ions times the charge per ion.
ϕ = ion density (ions per m3)
I q = ze = ionic charge (Coulombs per ion)
= qϕvd = ion valence times 1.6x10-19 Coulombs
A
• Plugging in drift velocity from above, we get:
I D
= qϕ (qE )
A kT
42
Derivation of resistivity
• Thus, the current density (coulombs per second per unit area is
just proportional to the electric field:
I q 2ϕD I ⎛ 1⎞
= E =⎜ ⎟E
A kT A ⎝ ρ⎠
• Solving for ρ we get:
kT
ρ= 2 = resistivity (Ωm)
q ϕD
43
Extra slides on derivation of
Fick’s first law
We will now use a similar approach to derive a macroscopic
description of diffusion – a differential equation that describes the the
flux of particles from the spatial distribution of their concentration.
N(x) is the number of particles in A is the area of the
a box (of length δ) at position x interface between
the boxes
N(x) N(x+δ)
x x+δ
1
2 N(x)
1
2 N(x + δ)
1
2 [ N(x)− N(x + δ)] is the net number of particles moving to
the right in an interval of time τ 44
Extra slides on derivation of
Fick’s first law
We can calculate the flux in units of particles per second
per area
1 1 N(x)
Jx = − [N(x + δ)− N(x)] N(x+δ)
Aτ 2
x x+δ
multiply by δ 2 / δ 2
δ 2 1 ⎡ N(x + δ) N(x) ⎤ Particles per unit volume
Jx = − ⎢ − ⎥
2τ δ ⎢⎣ Aδ Aδ ⎥⎦
Density - particles per unit volume
1
J x = −D [ϕ(x + δ)−ϕ(x)]
δ
Note: To get density (ions/m3) from molar
concentration (mol/L), you have to
∂ϕ
J x = −D multiply by NAx10-3. (NA is Avagadro’s
∂x Number = 6.02x1023)
45
MIT OpenCourseWare
https://ocw.mit.edu/
9.40 Introduction to Neural Computation
Spring 2018
For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.
