When we say that "cognition is information processing", this is a meaningless statement unless we know what "information processing" is all about. Early on in this chapter -- when symbols and symbol manipulation are being described -- Cooper is telling us what information processing is.

A second theme that emerges from the chapter comes from the fact that in addition to understanding "information processing", this kind of processing is actually a really cool tool -- we can build computer programs that simulate the information processing that we think might be going on in the heads of our subjects. So, important issues to consider revolve around the nature of the theories that are provided when we put this tool to use.

A third theme, introduced at the end of the chapter, is the notion of cognitive architecture. One of the most important things that you should get out of this course is the concept that the primary goal of cognitive science is specifying the cognitive architecture. You will see why this is important as the course progresses!

Computer simulation is a central tool of cognitive science. "But what sort of theories are amenable to simulation, and how do we go about producing such a beast?" (p. 24). Main theme of chapter is to show what a "cognitive scientific theory" looks like.

Cognitive scientific theories are often represented as box/arrow diagrams (NB: Boxologies!) This is not a flowchart, because boxes don't represent decisions. "In this form of diagram, boxes are used to represent postulated cognitive processes and arrows between those boxes used to represent communication, or flow of information, between those processes." Boxes are best viewed as representing modular processes (NB: this reminds me of Farah's notion of locality) which is the main topic of chapter 3. However, while the distinction between flowcharts and box/arrow diagrams is important, it is often ignored. Key idea -- boxology gets converted into a simulation by filling in the boxes with "appropriate computational processes".

What is the purpose of simulation? 1) Verifying the behavior claimed by or predicted by a theory. But, with respect to this goal, simulations are very focused with respect to domains of inquiry. "In order to interpret simulation results as behavior it is generally necessary to make assumptions about the processes not covered by the simulation" (p. 27). 2) Isolating the poorly developed aspects of a theory: "the act of implementing a theory as a computer program can force a degree of precision which is sometimes lacking in cognitive psychology." 3) Simulation is a medium that permits a theory to be fine tuned -- to fit data better, an "empirical approach to theory development."

(NB: Here we get a first look at the guts of what is often called "classical cognitive science.) High-level cognition appears to involve the explicit manipulation of symbols. What are symbols? What do we mean by "symbol manipulation"?

Representation In Symbolic Models. "Central to the enterprise of symbolic modeling is the assumption that there exist such things as mental representations and that those mental representations are structured, semantically interpretable, objects" (p. 28). Mental representations are typically viewed as being built from simple symbols, symbols which can (like atoms) be combined into more complex things. As a result, it is claimed that such systems are compositional and systematic. Why is this important? Because it points to a basic way that a finite machine might be capable of an infinite variety of behavior!

Models Of Symbolic Computation. What is symbol manipulation? It is the kind of thing done by a computing machine. For example, one very simple machine is a finite state automaton. Such a machine works by "taking one input symbol on each cylce and using this symbol and its current state to generate a new state." In other words, this is a stimulus-response machine.

Another example is a machine called a push down automaton. It is like a finite state automaton with an infinite memory -- but you can only access the top element from this memory. "The addition of a stack to a finite state automaton gives a push down automaton substantially greater computational power, in that it increases the range of tasks of which the automaton is capable" (p. 31). For instance, now it can deal with embedded structure (clauses).

Another example is a Turing machine. "A Turing machine (conceived by Alan Turing) has an infinite memory in the form of a tape. This tape can be read off, or written onto, and all locations on the tape are open for inspection at any time." This kind of machine is very powerful! "Any taks which may be specified in terms of a procedure that is known to terminate (i.e., an algorithm) can be performed by a Turing machine with an appropriate state transition function. Universal Turing machines, machines which emulate any other Turing machine given a tape with a description of another Turing machine and its input, can even be designed. This raises an important issue -- is the brain as powerful as a Universal Turing machine? (NB: Classical cognitive science says yes to this question!)

Another example is a von Neumann machine. It is like a Turing machine with random access memory. Because of this it is much more efficient -- but a von Neumann machine cannot solve problems that a UTM is unable to solve! (NB: "Program storage" is also true of UTM, and is not just a von Neumann machine construct!)

Another example is a production system, which is a potential cognitive computing device. It is a long term memory of "if--then" rules (productions) that manipulate symbols stored in a working memory. "A production system's processing cycle consists of two phases. In the 'recognize' phase, the processor finds all rules in long-term memory whose conditions are satisfied by the elements in working memory. In the 'act' phase, the system chooses one matching rule and executes its right-hand side, thus chaning working memory and/or performing some action." (NB: It too has the same competence as a UTM!)

All of the architectures described up to this point are classical (or symbolic) in nature. Connectionism is an alternative to classical architectures. "A radically different approach to computation is employed by models that compute through the simultaneous operation of simple communicating processing devices. This form of processing, often termed connectionism, takes its inspiration from neurophysiology" (p. 34). (NB: In point of fact, the extent of this "radical difference" is subject to a great deal of debate.)

A connectionist network is a set of interconnected processing units, that are analogous to neurons. PDP version of connectionism assumes distributed representations (instead of symbols!), where concepts are represented as a pattern of activity across a whole bunch of processing units.

There are many different classes of networks, such as an associative network that serves as a simple memory device.

Another example is a feedforward network, where a single originates in input units, gets processed by hidden units, and then eventually causes a response in a set of output units.

Another example is a recurrent network, which is like a feedforward network with memory. This kind of network has been proven to be at least as powerful as a finite state automaton. (NB: In fact, it and feedforward nets might also be as powerful as UTMs!)

One important property of networks is that instead of being programmed, they are taught. The book briefly describes two important kinds of learning -- the Hebb rule for associative networks, and the backprop rule for feedforward networks.

"Once trained, PDP networks often exhibit psychologically interesting properties such as interference and generalization" (p. 39). As well, PDP networks often degrade gracefully, and are damage resistant.

Some models are connectionist, but are not PDP. This is because they don't use distributed representations. A Classic example of this is a semantic network. Each note in this kind of network stands for a particular concept (and thus is a local representation). Connections between nodes represent a specific relationship between concepts (is-a, is-part-of, etc.).

Another example is a production system with spreading activation. The activation, as it spreads, affects the salience of items in the working memory of the production system.

Another example is an IAC network. In this kind of network, there is (again) activation that spreads among nodes. However, there is a set of connections that enforce lateral inhibition, producing a kind of processing that is often called "winner take all". "Such models are able to produce discrete behavior by including some mechanism for selecting the most active symbol. Behavior is then determined by this symbol" (p. 42).

Connectionism has a long history, though it waned for a while in the 1970s. New discoveries led to a resurgence of connectionist research in the mid 1980s until the present. Since this resurgence "an intense debate -- concerning connectionist modeling, symbolic modeling, their relation, and their role in cognitive science -- has been raging" (p. 45).

One issue in this debate is that connectionist models appear to be better than symbolic models at accounting for stuff like graceful degradation. However, the reverse is true for properties like systematicity and compositionality. "The response from the connectionist camp has been to engineer connectionist methods for representing structured information."

A second issue revolves around criticisms of connectionism. Networks are often viewed as being black boxes that cannot be interpreted, or as mere implementations (i.e. neuroscience -- not cognition!). (NB: New techniques for interpreting networks, developed in my lab, are really causing trouble for these kinds of criticisms.)

Other issues focus on figuring out the true nature of rules (or of symbols for that matter), and on whether a compromise between the two camps is possible.

One approach to a compromise is to create a hybrid model. "These models are based on the twein assumptions that a) neither symbolic nor connectionist techniques are inherently flawed, and b) the techniques of both forms of modeling are compatible." There are two general approaches to developing hybrid models. "First, hybrid models might be constructed by joining together separate symbolic and connectionist submodels.  Seondly, 'non-physically hybrid models' may be developed in which there is only a single physical system (i.e., just one symbolic system, or just one connectionist system, but not one of each), but hybridness is obtained by describing the system in both symbolic and connectionist terms."

Physically hybrid models are suited for tasks that can be decomposed into modules, some of which are symbolic, and others of which are connectionist. "Two issues are central to physically hybrid modeling: interfacing, or the method of communication between symbolic and connectionist subsystems, and modularity, or the decomposition of a task into subtasks." IAC network is one architecture that is well suited for dealing with these issues.

In a non-physically hybrid model, "a single system is described or interpreted in both symbolic and connectionist terms." One example of this is Touretzky and Hinton's connectionist production system.

"We need to look at the mind as a complete entity, and develop complete or 'unified' theories of cognition. This requires a theory of the cognitive architecture." (NB: We will see much better reasons for needing this kind of theory as the course progresses!) What is an architecture? "A cognitive architecture is  an arrangement of functional components (e.g., processors and buffers) together with a processing strategy. For a computer to achieve a given task, it must be supplied with a program. The architecture is just the basis for executing the program." (NB: For now, think of the cognitive architecture as the programming language for human thinking.)

Architectures are relatively fixed. The choice of architecture affects the kinds of functions that can be run easily. "One architecture might be very efficient at certain operations, but inefficient at others, whereas a second architecture may have the opposite characteristics."