University of Alberta Dictionary of Cognitive Science: Ryle's Regress

Ryle's Regress is a classic argument against cognitivist theories, and concludes that such theories cannot be scientific. The philosopher Gilbert Ryle (1949) was concerned with critiquing what he called the intellectualist legend, which required intelligent acts to be the product of the conscious application of mental rules. Ryle (p. 31) argued that the intellectualist legend results in an infinite regress of thought:

According to the legend, whenever an agent does anything intelligently,
his act is preceded and steered by another internal act of considering a
regulative proposition appropriate to his practical problem. [...] Must we
then say that for the hero's reflections how to act to be intelligent he must
first reflect how best to reflect how to act?  The endlessness of this implied
regress shows that the aplication of the appropriateness does not entail the
occurrence of a process of considering this criterion.

Variants of Ryle's Regress are commonly aimed at cognitivist theories. For instance, in order to explain the behavior of rats, Edward Tolman (e.g., 1932, 1948) found that he had to use terms that modern cognitive scientists would be very comfortable with. For instance, Tolman suggested that his rats were constructing a "cognitive map" that helped them locate reinforcers, and he used intentional terms (e.g., expectancies, purposes, meanings) to describe their behavior. This led to a famous attack on Tolman's work by Guthrie (1935, p. 172):

Signs, in Tolman's theory, occasion in the rat realization, or cognition,
or judgement, or hypotheses, or abstraction, but they do not occasion action.
 In his concern with what goes on in the rat's mind, Tolman has neglected to
predict what the rat will do.  So far as the theory is concerned the rate is
left buried in thought;  if he gets to the food-box at the end that is his
concern, not the concern of the theory.

Cognitive scientists must be constantly aware of Ryle's Regress as a potential problem with their theories, and must ensure that their theories include a principled account of how the (potentially) infinite regress that emerges from functional analysis can be stopped. This is why the identification of the functional architecture is one of the fundamental goals of cognitive science.

References:

Guthrie, E.R. (1935). The psychology of learning. New York: Harper
Ryle, G. (1949). The concept of mind. London: Hutchinson & Company.
Tolman, E.C. (1932). Purposive behavior in animals. New York: Century Books.
Tolman, E.C. (1948). Cognitive maps in rats and men. Psychological Review, 55, 189-208.