Some philosophers of cognitive science suggest that classical and connectionist models in cognitive science capture regularities at different levels (von Eckardt, 1993).  For instance, Von Eckardt suggests that if one considers distributed representations in artificial neural networks as being “higher level” representations, then connectionist networks can be viewed as being analogous to classical architectures.  This is because when examined at this level, connectionist networks have the capacity to input and output represented information, to store represented information, and to manipulate represented information. In other words, the symbolic properties of classical architectures may emerge from what are known as the subsymbolic properties of networks (Smolensky, 1988).  To say that network properties are subsymbolic is to say that the processing of networks involves mechanisms that cannot be captured in formal rules (but instead may be better captured by dynamic models, or stochastic mechanics).  However, use of the term “subsymbolic” also implies that there will be some mapping from network properties to classical rules.  For instance, it is typical to claim that symbolic models are approximate accounts of what goes on in networks; more accurate accounts require the vocabulary the captures the specific subsymbolic dynamics.

References:

1. Smolensky, P. (1988). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1-74.
2. von Eckardt, B. (1993). What Is Cognitive Science? Cambridge, MA: MIT Press.

(Added April 2011)