Leslie G. Valiant (Harvard University)
Abstract. The brain performs many kinds of computation for which it is challenging to hypothesize any mechanism that does not contradict the quantitative evidence. Over a lifetime the brain performs hundreds of thousands of individual cognitive acts, of a variety of kinds, most having some dependence on past experience, and having in turn long-term effects on future behavior. It is challenging to reconcile such large scale capabilities, even in principle, with the known resource constraints on cortex, such as low connectivity and low average synaptic strength, and with the requirement that there be explicit algorithms that realize these acts.
Here we shall describe model neural circuits and associated algorithms that respect the brain's most basic resource constraints. These circuits simultaneously support a suite of four basic model tasks that each requires some circuit modification: memory allocation, association, supervised memorization, and inductive learning of threshold functions. The capacity of these circuits is established by simulating sequences of thousands of such acts in a computer, and then testing the circuits created for the cumulative efficacy of the many past acts. Thus the earlier acts of learning need to be retained without undue interference from the more recent ones. The prospects for experimentally validating these algorithms for the brain will be discussed.
Hierarchical memory allocation to arbitrary depth has the added requirement that a stably controlled number of neurons be assigned to memories at every level. We give a mechanism for this that can be realized in a shallow feedforward network. We suggest that in the brain it is the hippocampus that performs this stable memory allocation.