Outer Product

An outer product is a procedure in linear algebra that combines two vectors (Banchoff & Wermer, 1992). Let a be a column vector with x entries, and let b' be a row vector with y entries. The outer product of these two vectors is D = ab' where D will be a matrix that will have x rows and y columns. Each entry in the matrix produced by computing the outer product arises by multiplying an entry in the first vector by an entry in the second vector. In general, the value in matrix D found at row q and column r is equal to the q-th entry of a multiplied by the r-th entry of b'. Note that the outer product requires the first vector being combined to be a column vector, and the second vector being combined to be a row vector. If the reverse is true, an inner product will be computed, producing a single number, not a matrix!

The inner product, like most basic operations of linear algebra, is important to mathematically describing various operations in connectionist networks (Jordan, 1986). A distributed associative memory is a system that stores associations between the patterns of activity in two different sets of processing units. The outer product of the two vectors representing these sets of activities computes the desired weight changes for this system if the Hebb learning rule is being used to perform the association (Dawson, 2004).

References:

1. Banchoff, T., & Wermer, J. (1992). Linear Algebra Through Geometry. New York, NY: Springer.
2. Dawson, M. R. W. (2004). Minds And Machines : Connectionism And Psychological Modeling. Malden, MA: Blackwell Pub.
3. Jordan, M. I. (1986). An introduction to linear algebra in parallel distributed processing. In D. Rumelhart & J. McClelland (Eds.), Parallel Distributed Processing, Volume 1 (pp. 365-422). Cambridge, MA: MIT Press.

(Added January 2010)