Neuronal dynamics

For a single neuronal layer behavior is stochastic in absence of stimuli, periodic and largely phase synchronized with presence.

Train bottom up pathways to target post-lateral influence values while top-down targets pre-lateral influence values

Instead of training N parameters on S steps of gradient descent slowly, train parameters individually for N steps. O(NS) > O(N)

In equilibrium,

B ≈ N ≈ S

Weight updates per batch O(1) instead of O(n)

Each of N gradient descent updates costs O(n). Total cost O(n^2)

With single most active assignment, only O(1) computations are run per optimization step. Total cost O(n)

Forward pass still costs