Distributed Associative Memory

A distributed associative memory is a kind of artificial neural network that consists of one set of input units and another set of output units. The network is fully connected -- every output unit receives a signal from every input unit. The purpose of the memory is to store an association between different pairs of input-output activations. Later, when only the input activity of one pair is presented to the input units of the network, the goal is for the signal produced by this input to reproduce the output activity of the pair in the output units.

The structure of this memory can be traced back at least as far as James (1890), and it was used to model a variety of human memory phenomena in the 1980s (e.g. Hinton & Anderson, 1981).

In spite of its popularity, the memory has several limitations, particularly when trained with the Hebb rule (Dawson, 2004), and only slightly improved when trained with the delta rule. These limitations are largely due to the fact that the activation function of the output units is linear. Nonetheless, the fundamental characteristics of this kind of network are preserved in more advanced networks that "tweak" learning rules to accommodate innovations like nonlinear activation functions.

References:

1. Dawson, M. R. W. (2004). Minds And Machines : Connectionism And Psychological Modeling. Malden, MA: Blackwell Pub.
2. Hinton, G. E., & Anderson, J. A. (1981). Parallel Models Of Associative Memory. Hillsdale, NJ: Lawrence Erlbaum Associates.
3. James, W. (1890). The Principles Of Psychology, Volume One. New York, NY: Dover Publications.

(Added January 2010)