Coarse Code

A coarse code is a concept that comes from connectionist cognitive science. It is an example of a distributed representation that has been discovered by some network trained to do a task. In a coarse code, an individual hidden unit represents a particular kind of property, but does so quite inaccurately. However, if the responses of a number of these units is pooled together, the result can be a very accurate representation. This is particularly true if there is variability across the hidden units, so that they measure different ranges of the property; at the same time, overlap between sensitivities of hidden units is required for coarse coding to succeed.

Examples of coarse coding can be found in networks that have been trained to make spatial judgements. In one network (Dawson, Boechler & Valsangkar-Smyth, 2000) each hidden unit represents locations in a fairly inaccurate one-dimensional map. When hidden unit responses are pooled, though, an accurate two-dimensional map is the result, because each individual 1D map is constructed from a different perspective. In another (Dawson & Boechler, 2007), individual hidden units serve as highly inaccurate compasses -- when pooled, though, they produce an accurate representation of the direction from one map location to another.

References:

1. Dawson, M. R. W., Boechler, P. M., & Valsangkar-Smyth, M. (2000). Representing space in a PDP network: Coarse allocentric coding can mediate metric and nonmetric spatial judgements. Spatial Cognition and Computation, 2, 181-218.
2. Dawson, M. R. W., & Boechler, P. M. (2007). Representing an intrinsically nonmetric space of compass directions in an artificial neural network. International Journal of Cognitive Informatics and Natural Intelligence, 1, 53-65.

(Added November 2009)