Year: 2014
Type: editorial
Abstract: By taking it as a given that utility maximization is a valuable tool for constructing theories of behavior—thereby sidestepping controversies about the reality of optimality in the natural world—this special issue brings into sharp focus the roles and varieties of bounds that must be invoked in any sensible theory of behavior involving utility maximization. The articles assembled do an admirable job of covering this territory. They provide examples from multiple disciplines of how optimization in the face of bounds leads to compelling accounts of observed behavior; they explain subtle ways that boundedness and optimization relate to one another; and they provide evidence that bounded optimality can form the basis of a principled approach to understanding behavior and the mechanisms that generate it. Broadly stated, the task is to replace the global rationality of economic man with the kind of rational behavior that is compatible with the access to information and the computational capacities that are actually possessed by organisms, including man, in the kinds of environments in which such organisms exist. The article by issue editors Richard Lewis, Andrew Howes, and Satinder Singh is a good place to take a step beyond Simon. Building on a formalization of bounded optimality from artificial intelligence, their framework describes four classes of possible explanations distinguished by the kinds of bounds considered and how the utility of an agent's behavior is assessed. The most comprehensive type of explanation, which they call Ecological-Bounded-Optimality, includes limitations on the sensory, motor, and cognitive capacities of the organism as well as bounds derived from the distribution of environments to which the organism is assumed to be adapted. Key to their approach is formalizing how processing constraints bound the selection of programs that can run on an organism's physical machinery as it interacts with its environment. By delimiting the programs, processing constraints delimit the subset of possible behavioral histories that the organism can generate. Instead of evaluating specific decisions or behaviors, the entire program is evaluated by evaluating the whole set of histories it is able to generate. While somewhat daunting on the first read, this article goes a long way toward clarifying the nature of explanations based on bounded optimality, and it helps to unify approaches involving top-down and bottom-up theories of both behavior and cognitive mechanisms. Other articles focus on the nature and implications of specific instances of constraints. Ulrike Hahn makes a compelling case that the explanation for some apparently irrational behavior lies in the fact that small-sample statistics are very different from the large-sample statistics that underlie normative theories. Because we experience the world through a “narrow window” imposed by attention and short-term memory limitations, our experience is better described by the statistics of small samples. As an example, Hahn looks at experience generated by repeatedly tossing an unbiased coin. Assuming we remember only the last few tosses (maybe about seven), the frequencies of seeing some subsequences within this narrow window are different from the frequencies of seeing others. (If you do not believe this, read her article!) Because we do not have direct experience with the long-run statistics, seemingly irrational conclusions about randomness, such as the gambler's fallacy, are understandable. Hahn's article explores other very intriguing implications of short-run statistics and reviews a variety of small-sample studies in the Judgment and Decision Making literature. The constraints on which Hahn focuses are due to limitations of an organism's attention and memory processes (although they could also be viewed, as she points out, as environmental constraints since they produce an “effective environment” consisting solely of short-run samples), but her article does not attempt to sort out details of just what perceptual and cognitive mechanisms an organism may be using. Philip Holmes and Jonathan Cohen tackle this kind of question by examining the speed-accuracy tradeoff when subjects are told to maximize reward rate in a two-alternative forced-choice task, where rewards are earned for correct responses. Since higher accuracy requires longer response times, the best reward rate is achieved by sacrificing accuracy for speed in a way depending on the exact parameters of the task. Holmes and Cohen focus on a particular class of mechanisms—leaky competing accumulators, a special case of which are drift diffusion models—and look at how human behavior deviates from reward-rate-maximizing behavior. Deviation from optimality could be due to the fact that subjects are solving some other problem (presumably close to optimally), such as focusing on accuracy despite their instructions to maximize reward rate; or they are falling short of solving the given problem due to processing limitations, such as difficulty in estimating the duration of time intervals. The key point is that these possibilities can be tested experimentally by incorporating these constraints into well-defined models and seeing which accounts better for the experimental data. A similar theme is taken up by behavioral ecologists Pete Trimmer and Alasdair Houston. Behavioral ecologists have long developed theories of animal behavior by appealing to optimality, assuming that evolution tends to maximize reproductive success in the environments that an animal's ancestors had to face—perhaps ecological rationality in its purest form. Emphasizing the fact that normative prescriptions, including those sensitive to ecology, do not tell us much about mechanisms, Trimmer and Houston argue—as do Holmes and Cohen—that we can gain understanding of mechanisms by using models of mental machinery to make testable predictions about departures from optimality. They call this the nascent field of “evo mecho.” Trimmer and Houston discuss an example, among others, from foraging theory that closely parallels the forced-choice task of Holmes and Cohen, only this version is with bees. Is it better for a bee to take time to discriminate between flower types to avoid trying to obtain nectar from “empty flowers,” or is it better for it to be a fast forager and approach all flowers in the same way? Like the signal-to-noise ratio in the human experiment, the frequency of empty flowers determines which strategy is the best. Trimmer and Houston emphasize that some behavior may be suboptimal even in the appropriate ecological setting due to limitations of the evolutionary process. Evolution might have become trapped in a local maximum of the fitness landscape, making an optimal mechanism out of its reach, or it may be unable to fine tune mechanism parameters to suite particular circumstances. Especially interesting are implications for the evolution of learning mechanisms. A suboptimal learning mechanism, such as one based on the Rescorla-Wagner/Least-Mean-Square rule, might be more easily “‘discovered’ by mutations and recombinations” than the optimal Bayesian rule because its parameters do not need to be precisely tuned to operate effectively, whereas the Bayesian rule is more sensitive to its parameter settings. The tradeoff between optimality and robustness is as much an issue for evolution as it is for engineering methods based on optimization. Peter Dayan addresses a somewhat different question: What kinds of adaptations mitigate the negative consequences of various bounds? He contrasts algorithms capable of computing optimal (in a normative sense) behavior with simpler algorithms that often perform almost as well. What features do these simpler algorithms have that makes this possible? In one of his examples, Dayan uses the new label “Pavlovian pruning” for the idea that in searching the paths of a decision tree for an optimal plan, some branches are reflexively “pruned” upon encountering a sufficiently high cost, thus, making it impossible to discover that continuing the search along that branch might actually yield an optimal plan in which the early cost is outweighed by later rewards. The term “Pavlovian” is appropriate because Pavlovian responses often persist despite their overall negative consequences for obtaining reward. Dayan's article blends the perspectives of multiple disciplines in unorthodox but thoughtful ways. For example, the notions of model-free and model-based decision making from artificial intelligence closely correspond to the more traditional notions of habitual and goal-directed behavior, but they bring with them connections to specific computational architectures and algorithms that are lacking in the more time-honored usage. The reader not familiar with concepts from machine learning may find Dayan's intensely computational article difficult, but attending to its details will provide a glimpse into the modern blend of artificial intelligence, animal behavior, and neuroscience that is showing promise in illuminating issues in behavioral science. Implicit in some articles and explicit in others is the idea of accounting for the cost of computation in assessing utility. Analyses of speed-accuracy tradeoffs in humans and in animals implicitly involve the cost of computation because more computation takes more time. Trimmer and Houston note that in models of animal foraging, the cost of computation appears as increased risk of predation. Holmes and Cohen invoke the cost of cognitive control in assessing strategies for coping with changing conditions. The formalism of Lewis and co-authors is general enough to include computational cost because that cost contributes to the utility assessment of behavioral histories, with time and even energy costs playing a role in determining the histories that a program can generate and how they are evaluated. The cost of computation is treated explicitly in the contribution of Joseph Halpern, Rafael Pass, and Lior Seeman. Using a perspective from game theory (although specialized to single agents), in one of their abstractions, each play is represented by the choice of a Turing machine, and the player is “charged” according to the complexity of the machine chosen. They argue that the cost of computation can explain several biases observed in human decision making. For example, the “first-impression-matters bias” can be explained by assuming that there is a non-negligible cost for absorbing new information. This article illustrates how the theory of computation can add its own perspective to questions about optimality and bounds. Taken as a whole, this collection provides a terrific opportunity to go beyond the confines of single disciplines and simplistic views of the role of optimality arguments in theories of mind, brain, and behavior.
Source: Topics in Cognitive Science
DOI: 10.1111/tops.12090
Author: Andrew G. Barto
Institution: University of Massachusetts System
Language: English
Subfield: General Decision Sciences
Field: Decision Sciences
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W1970734895