Linear System

If a system is linear, then its whole behavior is exactly equal to the sum of the behaviors of its parts (Luce, 1999).  The standard pattern associator -- a distributed memory trained with the Hebb rule, with a set of input units connected directly to a set of output units (Dawson, 2004, Chapter 9) -- is an example of such a system.  Each output unit in the standard pattern associator computes a net input, which is the sum of all of the individual signals that it receives from the input units.  The activity of the output unit is exactly equal its net input.  In other words, output activity is exactly equal to the sum of input signals in the standard pattern associator.  In order to increase the power of this type of pattern associator a nonlinear relationship between input and output must be introduced a relationship where output activity is a nonlinear transformation of net input.

References:

1. Dawson, M. R. W. (2004). Minds And Machines : Connectionism And Psychological Modeling. Malden, MA: Blackwell
2. Luce, R. D. (1999). Where is mathematical modeling in psychology headed? Theory & Psychology, 9, 723-737.

.(Added October, 2010)