Learning Rule

Learning rules, for a connectionist system, are algorithms or equations which govern changes in the weights of the connections in a network (Bechtel & Abrahamsen, 2002). One of the simplest learning procedures for two- layer networks is the Hebbian Learning Rule, which is based on a rule initially proposed by Hebb (1949). Hebb's rule states that the simultaneous excitation of two neuron results in a strengthening of the connections between them. More powerful learning rules (Rumelhart & McClelland, 1986)are learning rules which incorporate an error reduction procedure or error correction procedure (e.g., delta rule, generalized delta rule, back propagation). Learning rules incorporating an error reduction procedure utilize the discrepancy between the desired output pattern and an actual output pattern to change (improve) its weights during training. The learning rule is typically applied repeatedly to the same set of training inputs across a large number of epochs or training loops with error gradually reduced across epochs as the weights are fine-tuned.

References:

1. Bechtel, W., & Abrahamsen, A. A. (2002). Connectionism And The Mind : Parallel Processing, Dynamics, And Evolution In Networks (2nd ed.). Malden, MA: Blackwell.
2. Hebb, D.O. (1949). The organization of behavior. New York: Wiley.
3. Rumelhart, D.E., & McClelland, J. L.(1986). Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1: Foundations. Cambridge, MA: MIT Press.

(Revised February 2010)