Inner Product

An inner product, sometimes known as a dot product, is a procedure in linear algebra that combines two vectors (Banchoff & Wermer, 1992). Let a' be a row vector with x entries, and let b be a column vector with x entries. The inner product of these two vectors is y = a'·b where y will be a single number. An inner product will always be a single number (a scalar), because the inner product is simply a sum of multiplications. That is, the inner product of a' and b is the product of the first entries in each vector, plus the product of the second entries in each vector, and so on until all products are computed and added together. Note that the inner product requires the first vector being combined to be a row vector, and the second vector being combined to be a column vector. If the reverse is true, an outerproduct will be computed, producing a matrix, not a single number!

The inner product, like most basic operations of linear algebra, is important to mathematically describing various operations in connectionist networks (Jordan, 1986). Let vector a' represent the set of connection weights between a set of input units and another processing unit (e.g. an output unit). Let vector b represent the activities of the various input units. The inner product a'·b is the net input for the output unit. Indeed, the inner product is the standard equation for computing net input in most connectionist networks.

References:

1. Banchoff, T., & Wermer, J. (1992). Linear Algebra Through Geometry. New York, NY: Springer.
2. Jordan, M. I. (1986). An introduction to linear algebra in parallel distributed processing. In D. Rumelhart & J. McClelland (Eds.), Parallel Distributed Processing, Volume 1 (pp. 365-422). Cambridge, MA: MIT Press.

(Added January 2010)