Experiments with an artificial model that mimics real neurons. "The neural simulation now includes these behaviors: A precise neuron action threshold, a precise action pulse waveform in axons, a variable action recovery (refractory) period, inhibitory neurons and synapses, rapid rise / slow drop of potential in synapses, temporal summation of pulses in synapses, spatial summation of pulses in dendrites, and long-term potentiation (LTP) and depression (LTD) in synapses."
His simulation uses neurons connected in an extremely regular, symmetric fashion, and he used a 1-pixel-per-neuron representation scheme to watch the simulations, with color codes for the various neuron states.
What has he learned?
Neurons have a precise action threshold. "This behavior is well understood and indicates that the brain is not a purely analog system. This aspect is more like a digital behavior. A neuron can only fire or not fire. There is no partial credit for an input potential just below the action threshold. It is ironic that many public ANNs (simulations on digital computers) attempt to be fully analog." That's because backpropagation, an algorithm based on the chain rule in calculus, needs for everything to be fully differentiable.
"The action pulse that travels through the axon also looks like a digitally generated square wave. It switches on very precisely, has a precise duration (approx. 1 msec), and then switches off very precisely -- with one exception: when the action potential switches off, it goes negative for short period. It is interesting that this is also a digital behavior."
"Once a neuron fires, it normally cannot fire again for approx. 10 msec."
"Neuroscientists have observed some neurons send excitatory (positive) pulses to other neurons and cause more activations, and some send inhibitory (negative) pulses and suppress activations."
"The complex electro-chemical processes in the synapses change the axon pulse dramatically. It is typically reduced in strength (by 95%!), slightly delayed, and no longer looks like a digital square wave."
"Time-based behavior occurs when multiple action pulses pass through a single synapse in rapid succession, close enough to overlap."
"This time-based behavior was discovered in 1966 and is still considered to be one of the main mechanisms supporting long-term memory and learning in the brain."TIME FOR AI TO LEARN
by Ed Rusis – Chief AI Research Scientist, Dendrite Software (Feb. 14, 2021) This neural network simulation demonstrates that in nature… time is a critical dimension of memory and learning.www.linkedin.com