Hebbian Learning Rule

The Hebbian Learning Rule is a learning rule that specifies how much the weight of the connection between two units should be increased or decreased in proportion to the product of their activation. The rule builds on Hebbs's 1949 learning rule which states that the connections between two neurons might be strengthened if the neurons fire simultaneously. The Hebbian Rule works well as long as all the input patterns are orthogonal or uncorrelated. The requirement of orthogonality places serious limitations on the Hebbian Learning Rule. A more powerful learning rule is the delta rule, which utilizes the discrepancy between the desired and actual output of each output unit to change the weights feeding into it.

References:

1. Bechtel, W., & Abrahamsen, A. (1993). Connectionism and the mind: An introduction to parallel processing in networks. Oxford, UK: Blackwell.
2. Hebb, D.O. (1949). The organization of behavior. New York: Wiley.
3. Rumelhart, D.E., & McClelland, J. L.(1986). Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1: Foundations. Cambridge, MA: MIT Press.