University of Alberta Dictionary of Cognitive Science: Parallel Distributed Processing

Parallel Distributed Processing (PDP) models are a class of neurally inspired information processing models that attempt to model information processing the way it actually takes place in the brain.

This model was developed because of findings that a system of neural connections appeared to be distributed in a parallel array in addition to serial pathways. As such, different types of mental processing are considered to be distributed throughout a highly complex neuronetwork.

The PDP model has 3 basic principles:

the representation of information is distributed (not local)
memory and knowledge for specific things are not stored explicitly, but stored in the connections between units.
learning can occur with gradual changes in connection strength by experience.

These models assume that information processing takes place through interactions of large numbers of simple processing elementscalled units, each sending excitatory and inhibitory signals to other units. (Rumelhart, Hinton, & McClelland, 1986, p. 10)

Rumelhart, Hinton, and McClelland (1986) state that there are 8 major components of the PDP model framework:

a set of processing units
a state of activation
an output function for each unit
a pattern of connectivity among units
a propagation rule for propagating patterns of activities through the network of connectivities
an activation rule for combining the inputs impinging on a unit with the current state of that unit to produce a new level of activation for the unit
a learning rule whereby patterns of connectivity are modified by experience
an environment within which the system must operate

References:

Rumelhart, D.E., Hinton, G.E., & McClelland, J.L. (1986). A general framework for parallel distributed processing. In D. E. Rumelhart, J. L. McClelland, and the PDP Research Group (Eds.). Parallel distributed processing: Explorations in the microstructure of cognition. Vol. 1: Foundations. Cambridge, MA: MIT Press.