Adaptive Individuals in Evolving Populations
Models and Algorithms

Introduction

Inside this Book

Series

Pub. Date

Publisher

Order Info

By Richard K. Belew and Melanie Mitchell

1. Book Overview: Form and Content

Even the simplest creature is marvelous to observe as it transforms itself to better match the environment in which it finds itself. How is such adaptation accomplished? How much of this capability should be attributed to the particular individual we happen to be observing, how much to its species, and how much to the inclusive evolutionary processes wedding all life to this planet? How did the elaborate individual learning we find in more complex organisms evolve? Once in place, does an increased individual capacity for adaptation alter the selective pressures causing the species to adapt to its niche?

This book has grown out of a workshop organized to address questions like these. The meeting was sponsored by the Santa Fe Institute and held at Sol y Sombra in Santa Fe, New Mexico, during July, 1993. It brought together a group of about 20 scientists from the disciplines of biology, psychology, and computer science, all studying interactions between the evolution of populations and individuals' adaptations in those populations, and all of whom make some use of computational tools in their work.

The questions that brought us together have been addressed by scientists and scholars for centuries. An excellent example is the "Baldwin effect" (a phenomenon identified by the psychologist J. Mark Baldwin exactly a century ago) that arises repeatedly in many modern computer simulations. Despite the potentially rich historical overtones offered by literature like this, it too often appears that the modern enterprise of science requires successful scientists to become a sort of "write-only memory," writing too much and not reading enough. To combat this tendency and acknowledge the scholarship that precedes our own, a collection of "classic" papers addressing these questions was circulated to all participants as preparation for our discussion.

Here we have reprinted eleven of these seminal classic readings, all addressing interactions between individual adaptation and population evolution. These texts include Baldwin's original paper (Chapter 5), for example, and range from a chapter from Lamarck's Zoological Philosophy of 1809 (Chapter 4) to the very brief 1988 paper by Hinton and Nowlan, "How Learning Can Guide Evolution" (Chapter 25), which served to awaken computer scientists to the Baldwin effect. Our selection was further refined by restricting it to papers that are less widely accessible than they deserve. It is for this reason only that centrally relevant papers (e.g., Dennett's [1981] "Why the Law of Effect Won't Go Away," or Plotkin's [1988] "Learning and Evolution") are not included. For many of the reprinted classics, we provide a specially written preface that helps to place this work in the context of the modern research agenda. [1]

In the face of this prior scholarship, the issue then becomes how these classic questions have been changed by knowledge gained in the interim, and what new insights we might gain from modern techniques, particularly computational ones. The other papers in this volume are based on presentations made during the meeting and the discussions that followed them.

Much of this discussion concerned the interpretation of results derived from one discipline by those outside the discipline. For example, what lesson can an ecologist draw from a computational analysis of function optimization? One consequence of this interdisciplinary dynamic [2] is that each paper was reviewed by one or more workshop participants, typically from disciplines different than that of the author. These papers are also preceded by short prefaces, written by the reviewer(s). The goal of the preface to each "modern" (versus classic) paper is to explain the interdisciplinary relevance of the ideas and results of the paper. We hope these gentle introductions will provide readers outside the authors' discipline with an appreciation of each contribution's value.

Figure 1. The book is divided into three parts corresponding to primarily biological, psychological, or computational themes. Each part begins with an overview chapter, and readers with backgrounds in one of these three areas may find their respective overview chapter the best place to begin.

With overviews, classic reprints, and new work, target readings and prefaces, this book's form is admittedly complex; the issues it attempts to address are no less so. Obeying the maxim that a picture is worth an entire chapter of words, we have attempted to capture the themes of the workshop and of this volume in a single diagram or "logo" that appears on the book's frontispiece and in Figure 1. First, this graphic reflects a sort of self-similarity between adaptive systems at two different levels: individuals (the multiple small blobs) capable of certain malleable deformations and the population (single large blob) in which they are contained are both capable of being plastic or adaptive.

Next, the diagram captures our fundamentally interdisciplinary approach. As mentioned above, the core issues we address are not new, but our attempt to attack them with a "pincer movement" using three distinct "armies" corresponding to the biological, psychological, and computational perspectives may be. We imagine these three perspectives as green, red, and blue spotlights, respectively, each highlighting salient aspects of the phenomena. The biologists at the workshop were especially focused on adaptation at the population level and on the environment in which this population finds itself. The psychologists were especially concerned with cognitive activity and plasticity in individuals. The computer scientists were most concerned with commonalties across these two levels--commonalties that will help in developing adaptive and evolving computer systems.

Given our (RB and MM) backgrounds as computer scientists, it is no surprise that the workshop's participants shared interest in and experience with computation. The purposes toward which the computational tools are applied can be quite different, however. The final theme captured by the logo (and also part of our title), then, is the distinction between models and algorithms. Briefly, we view "models" defined as tools operating as an extension of the enterprise of biology, psychology, and other natural sciences. "Algorithms," on the other hand, are tools designed to do some job for somebody; they are engineered artifacts whose success is measured in terms of effectiveness and efficiency in performing the given task. This dichotomy is far from pure; in Chapter 24 we address a number of complications to this simple picture, especially as they suggest research issues for computer science.

2. Major Themes of This Book

The readings in this book have a number of themes in common. All of them deal in some way with questions concerning adaptation: what is it, at what levels in living systems does it operate (e.g., genetic, phenotypic, and population levels), and how does adaptation at these various levels affect adaptation at the other levels?

Herbert Simon would have us take adaptation to be a sine qua non for any cognitive system:

"Cognitive science is... a fundamental set of common concerns shared by... disciplines concerned with systems that are adaptive--that are what they are from being ground between the nether millstone of their physiology or hardware, as the case may be, and the upper millstone of a complex environment in which they exist" (Simon, 1980).

As used by Simon and others, "adaptation" connotes not only the capacity for change, but the additional requirement that this change represents an improvement in "fit." As will become clear in a number of the readings, establishing a clear criterion by which such improvements can be measured becomes a difficult issue on its own, and so the term "plasticity," which refers more simply to "flexibility; capacity for change" (Gordon, 1992, p.255), is sometimes appropriate. [3] With this relaxation, we immediately make contact with evolutionary concerns, as Darwin himself noted: "I speculated whether a species very liable to repeated and great changes of conditions might not assume a fluctuating condition ready to be adapted to either condition" (Darwin, 1881, from Gordon, ibid.).

Note the important move, however, from considering plasticity to be a trait of individuals to one of populations. For populations, plasticity becomes "substantial individual variation among members of the same species" (Brauth, Hall, and Dooling, 1991, p.1). Common to both individual and population plasticity is that a capacity for change is implied. The comparison for individuals considers the same individual at two points in time; a plastic individual is one that exhibits a wide range of variability during its lifetime. A population's plasticity, on the other hand, requires comparison across members of the same species; a plastic species is one that exhibits wide phenotypic variation.

The methodological difficulties involved in assessing the range of phenotypic variation, particularly as it may be sensitive to environmental variation, has been a fundamental obstacle to the investigation of the role of plasticity in evolutionary theory (Mayr, 1982). The concept of "norms of reaction" (illustrated in Figure 2) has proven an important conceptual device in understanding this relationship. Imagine some range of environmental variation, and a genotype that exhibits a range of phenotypic variations in the face of these environmental variations. The genotype can then be viewed as a function, transforming environmental variables to phenotypic traits. This perspective helps to suggest a relationship between genotypic characteristics and environmental variation that together give rise to a particular phenotypic form, as well as variants in form that we might observe. The sticky, sticky issues of just how much of the variability we see across individuals should be ascribed to heritable traits that we associate with genetic factors, and how much to shared environmental commonalties will, in natural situations, be very hard to determine. In some carefully designed laboratory experiments we may be able to distinguish between these factors. In our computational simulations making these distinctions is trivial: by virtue of the program's construction we can know precisely the variables corresponding to the "genotypic" and "environmental" factors.

Figure 2. Norm of reaction.

Under this general rubric of adaptation and plasticity in individuals and in populations, some common threads tying the readings together are:

How direct are the interactions coupling genetic and phenotypic adaptations--i.e., interactions between adaptations that occur as a result of changes to the genome over evolutionary time, and those that occur via plasticity during an organism's lifetime?
In what ways should the behavior of an organism (as contrasted with its morphology or physiology) be viewed as a key phenotypic trait that is acted on by evolution?
More specifically, in what ways does an individual's learning behaviors during its lifetime interact with the evolution of its species?
To what extent do behaving/learning organisms effect a kind of "natural selection" on their environments, thereby producing a symmetric complement to the (more conventionally considered) selective forces that their environments apply to them?
To what extent are learning (of an individual over its lifetime) and evolution (of a species over many generations) two sides of the same coin---i.e., two manifestations (at different time scales) of a similar process?

In the following we give an overview of these questions and the chapters of this book that address them.

2.1 Direct Versus Indirect Interactions Between Genotypic and Phenotypic Adaptations

Many of the readings in this book concern the types of effects, both direct and indirect, that genotypic adaptations can have on phenotypic adaptations, and vice versa. The excerpt we reprint from Lamarck's Zoological Philosophy (Chapter 4) is one of the earliest discussions of this theme. As is well known, Lamarck proposed (long before the mechanisms of heredity were discovered) that phenotypic traits acquired by an organism during its lifetime can be passed on--directly--to the organism's offspring. One of the hallmarks of modern genetics is its rejection of this hypothesis. It is almost universally accepted among biologists that "reverse transcription" of acquired phenotypic traits back into the heritable genotype does not occur.

Figure 3 presents a simple, conventional perspective of the relationship between genotype and phenotype. We begin with a particular genotype at some time $G(t_0)$. According to Weismann's doctrine (e.g., see Weismann, 1893), sequestration of the germ line insulates subsequent phenotypic changes from evolution of the genotype. Following Elman et al. (in press), we will use the term development to refer to all types of within-lifetime change in individuals, but then distinguish between two important subcategories. The first type, maturation, refers to the process of transforming genotype into phenotype, the cause of which we primarily attribute to the genetic code. This gives rise to a neonate phenotype at a somewhat later time $t_1$. Learning is the second type of individual change, the cause of which we attribute primarily to interactions with the environment. This gives rise to an "adapted" phenotype that we consider at a still later time $t_3$. It is important to note that "maturation" and "learning" are idealized analytic categories, acting as poles for the full spectrum of developmental changes due to mixtures of genetic and environmental influence; most biological examples of developmental change will be extraordinarily complicated mixtures of the two. The final component of this simple diagram is an explicit blockage of the direct inheritance Lamarck proposed. [4] That is, our theories must disallow the genetic inheritance of characteristics acquired during the lifetime of an individual.

If only the story were as straightforward as this figure makes it seem! As our brief excerpt of Lamarck (Chapter 4) documents, this distinguished biologist and keen observer had many parts of the evolutionary story right 50 years before Darwin, and good reasons for the confusion that henceforth became his primary legacy. As suggested by Figure 4, the fact is that a common environment mediates the direct causal linkages captured in Figure 3. A range of other indirect interactions between genotype and phenotype then become possible. These include:enskip environmental influences on the maturational process transforming genotype to phenotype; the learning changes that shape a phenotype during its lifetime; the effects of its behaviors on the environment; and long-term (indirect!) effects these actions by individuals can have on populations.

Figure 3. Genotype/phenotype separation: standard view.

Figure 4. Genotype/phenotype interactions.

A good example of a phenomenon requiring more sophisticated analysis is provided by the recent experiments of Cairns Smith et al. (Cairns Smith, Overbaugh, and Miller, 1988; Foster, 1993). They have shown that E. coli are capable of unusual adaptations to characteristics of their lactose growth medium. A fundamental tenet of the neo-Darwinian synthesis is the independence of the sources of genetic variation from the selective forces subsequently applying to phenotypes. Yet recent laboratory data suggests that, at least in some simple bacteria, the genotype seems able to "adaptively mutate" and be responsive to environmental variability. The potential impact of mechanisms of adaptive mutation on a more complete theory of the interacting mechanisms of variation and selection are therefore profound:

"The discovery that cells use biochemical systems to change their DNA in response to physiological inputs moves mutation beyond the realm of "blind" stochastic events and provides a mechanistic basis for understanding how biological requirements can feed back onto genome structure...there is no unicorn in the genomic garden. But we have found a genetic engineer there, and she has an impressive toolbox full of sophisticated molecular devices for reorganizing DNA molecules" (Shapiro, 1995, p.374).

It is important to be clear that any such interactions must be indirect, and not of the sort proposed by Lamarck. We do not consider ourselves Lamarckian, nor to our knowledge do any of our contributors. All the explanations we consider are in basic accordance with the fundamental components of the neo-Darwinian explanation suggested by Figure 3; no new magical mechanism is proposed that might directly communicate information gained by the phenotype directly to the genotype. Much of the work in this book is motivated by a belief in the existence of plausible, non-Lamarckian mechanisms by which information acquired by phenotypic adaptations can eventually become genetically encoded. The Baldwin effect is but one example.

It is unfortunately true, however, that many of the phenomena that will most interest us are often confused with Lamarckianism. Consider:

"Piaget was greatly influenced by Waddington, but he went well beyond support for Waddington's notion of the exploitative system; he championed the `Baldwin Effect' and other rather Lamarckian notions by which behavior might directly affect the genotype--and for which no known mechanisms exist" (Plotkin, 1988, p.143, emphasis added).

It is not clear what notions Plotkin intends to preclude as "rather Lamarckian"; guilt by association seems sufficient! In his discussion of the Baldwin effect (Chapter 8 in this volume) Simpson makes a similarly confusing reference to "neo-Lamarckian," apparently as a parallel to the very well defined and useful "neo-Darwinian." Even Waddington seemed confused by the Baldwin effect, despite the fact that his mechanism of genetic assimilation shares a number of features with it.

We hope re-publication of some of the seminal papers at the heart of this confusion will help to clarify these critically important issues. Here we reprint readings by Baldwin, Lloyd Morgan, Waddington, Piaget, Simpson, and Bateson (Chapters 5, 6, 7, 19, 8, and 9, respectively) on this topic. Bateson (Chapter 9) perhaps goes furthest, arguing that, setting aside the impossibility of reverse transcription, the complexity and "economics of somatic flexibility" in organisms makes pure Lamarckianism unworkable, but gives an advantage to "simulated Lamarckianism" of the sort described by the Baldwin effect. From a computational perspective, Hinton and Nowlan's paper (Chapter 25) presents a highly simplified model of the Baldwin effect in order to demonstrate its plausibility. Among the contributed chapters, Parisi and Nolfi (Chapter 23) also demonstrate the Baldwin effect investigated, though in a richer, computational setting, and give a more extensive analysis of the underlying mechanisms. Hightower, Forrest, and Perelson (Chapter 11) examine the possibility of Baldwin-like effects in the immune system--another biological system in which adaptations occur both on evolutionary and within-lifetime time scales.

A number of these authors will be seen to be championing the Baldwin effect or its relatives, and all without recourse to Lamarckian magic. The critical issue seems to be just how strong the causal but indirect role environmentally mediated mechanisms can play in the evolutionary process. A central goal of this book is to get beyond a simple knee-jerk rejection of Lamarck's direct inheritance, so as to consider indirect ways by which phenotypic change can influence evolution.

2.2 Behavior as a Phenotypic Trait

There seems to be an inevitable bias in evolutionary theory toward things that fossilize well--morphological structures. A message sounded loudly by ethologists, however, is that behaviors too count as first-class phenotypic traits. Moreover, behaviors "close the loop" between an organism and its environment. It may be possible to describe a behavior in terms of physiological changes to an organism (e.g., range of limb movement), but without also describing the action's effect on the world in which the organism naturally finds itself, these physiological characteristics are meaningless. As several recent critiques of early artificial intelligence methods have made clear, this is as true of high-level, cognitive behaviors performed by humans as it is of amoebic locomotion. Planning that is "reactive" to environmental characteristics is simpler and more effective than forcing an agent to maintain an internal, completely consistent world model (Agre, 1995). Most intelligent behaviors are "situated": they must be understood in terms of the environmental and cultural context in which they occur (Suchman, 1987; Hutchins, 1995). In short, behavior weds organisms to their environments in a way that is critical to a full understanding of what it means to be adaptive (see Section 2.4).

Behavior will prove a particularly important class of traits to us in this book because behaviors are especially plastic, and because they play a particularly powerful evolutionary role. Their range of variability, relative to morphological changes, is pronounced. Lorenz (1973) describes this flexibility in terms of "gaps" in behavioral chains, places in the program controlling behavior in which a range of alternatives can be filled in by individuals of particular environmental situations. Mayr is often identified with a particularly strong account of behavior's evolutionary role:

"Many if not most acquisitions of new structures in the course of evolution can be ascribed to selection forces exerted by newly acquired behaviors. Behavior thus plays an important role as the pacemaker of evolutionary change" (Mayr, 1982, p.612).

Because behavioral traits are so hard to quantify, biologists typically consider behaviors with especially direct connections to selective fitness, such as foraging. The papers by Shafir and Roughgarden; Menczer and Belew; Parisi and Nolfi; and Miglino, Nolfi, and Parisi (Chapters 12, 13, 23, and 22, respectively) fall into this category. This last paper highlights several distinct, hierarchically nested levels--genotype, structural neural network, functional neural network, potential behaviors, actual behaviors--at which evolution may operate in a behaving organism, even when selection acts upon only the last.

Psychologists typically consider much more elaborate behaviors with less direct connection to selective pressures. One goal for our interdisciplinary discussion is to relate the more sophisticated computational models coming from cognitive science to the evolutionary issues of central concern to biologists. The ways in which this can be done are discussed in detail in the psychology overview, Chapter 14. The critical cognitive ability to predict future events is a topic of special concern in the papers of Zhivitovsky, Bergman, and Feldman (Chapter 10), and Parisi and Nolfi (Chapter 23).

2.3 Interactions Between Learning and Evolution

The benefits of individual plasticity to evolutionary adaptation seem self-evident: in any sufficiently complex, nonstationary environment, an individual able to adapt to changes in the environment during its lifetime is surely more fit than one that cannot. There are evolutionary costs, however, that must also be associated with this same plasticity, and a full analysis must consider both the costs and benefits as they apply to any particular species to understand the net evolutionary impact of individual adaptation. Here we will mention only some of the most important costs and benefits of learning with respect to evolution (Johnston's paper, Chapter 20, provides a much more extensive analysis), and then discuss the even more complicated interactions related to the evolutionary origins of learning.

Figure 5. Environmental regularities.

Learning provides a selective advantage

If we do as Herbert Simon says (above), and treat the problem of adaptation as one of identifying and exploiting environmental regularities, we can model a very simple version of this in terms of a one-dimensional time series (e.g., daily temperature values). Dynamical systems analysis tells us that these regularities can occur at many different time scales (e.g., days, years, millenia), and that their pattern may be arbitrarily complex. We might hope that an adaptive system is capable of finding regularities in such time series and exploiting them (via, for example, seasonal variations in coat thickness, burrowing behaviors, or migration patterns).

An abstracted time series and a simplified version of the relationship between evolutionary and learning systems is shown in Figure 5. First consider evolution, plodding along. Its "sampling" of the environment (denoted in the figure by short vertical bars) occur in the form of generations, and track those glacial changes occurring slowly enough that a species' gene pool can follow. [5] Within each generation, many individuals each experience spatially and temporally localized environments (denoted by more closely spaced but disjoint sets of vertical bars). We can easily imagine that, as in the figure, some frequencies of environmental change occur too quickly for evolutionary adaptation. In such cases learning may become another adaptive process by which individuals are transformed in response to regularities of the local environment they experience. For example, if temperature changes too rapidly for evolution to track, individuals can learn to predict short-term temperature changes during their lifetime and perhaps control their environments, by learning to burrow or hibernate when it is too cold outside. Such individual adaptations are in addition to any evolutionary changes their species may make.

In Piaget's words, useful behavioral adaptations are those:

"...that characterize varieties of behavior which naturally facilitate survival... [and] serve to increase the powers of the individual of a species by putting greater means at their disposal" (Piaget, 1978, p.xix).

Waddington has argued more broadly that since learning can be expected to increase the variance of individuals, and since variance across populations is evolution's primary grist, individual plasticity should speed up evolution. Ways in which individuals' adaptations can help to "inform" the evolutionary process are addressed by many of the papers in this collection, most centrally in the papers by Hart and Belew; Littman; and Zhivitovsky, Bergman, and Feldman (Chapters 27, 26, and 10, respectively).

Learning can also constitute a selective disadvantage

There are equally good arguments as to why learning, and plasticity in general, might slow down the evolutionary processes shaping populations. Sewall Wright (1980) observed that individual plasticity can effectively hide desirable phenotypic traits by allowing them to become temporarily achieved. That is, desirable phenotypic traits can be effectively masked because they can be achieved temporarily by some individuals via learning. Evolution gets less information concerning selective pressures in a particular environment. Gordon summarizes the argument:

"Of a range of phenotypes, only some are optimal. If the production of phenotypes were irreversible, not subject to change, then selection could choose the best ones and eliminate the others. But plasticity allows less than optimal organisms to slip out from under selection's heel, by taking a more optimal form temporarily" (Gordon, 1992, p.262).

It is also possible to point to fairly specific reproductive costs associated with learning. Most concrete are the additional metabolic costs associated with developing and maintaining the machinery for learning; the vertebrate central nervous system is an excellent example of both an advanced substrate for learning and its attending metabolic demands. A learning juvenile is also less well prepared for the world, and hence creates an additional "load" on the parents that must rear it (Cecconi, Menczer, and Belew, 1996). The same environmental and experiential factors to which a plastic individual must be sensitive exposes it to concomitant risk that the necessary environmental conditions for successful development will not prevail. The paradoxical balance of such risks with the equally obvious benefits of plasticity with which we began, sets the stage for the more refined analysis this book attempts.

The evolution of learning

Prerequisite to an understanding of the potential effects, facilitory or inhibitory, of learning on evolution, it is useful to consider the effects of evolution on learning, i.e., the evolution of the original learners, and the subsequent evolution of increasingly powerful learning capabilities. Bateson's article (Chapter 9) is one of the few to address such questions. If we view genetics as one adaptive "circuit" in the interior of a second adaptive system comprised of learning individuals, Bateson argues that there exist both "centrifugal" and "centripetal" forces shifting a "locus of control" to and fro between genetic and learning systems. Further, he suggests that the long-term tendency will be in favor of the centripetal force "affirming" learned behaviors via "genetic assimilation" (Waddington's term; see Chapter 7) in exactly those cases where environmental conditions remain constant long enough for this to occur. While Bateson is the first to acknowledge that "To speculate about problems so vast is perhaps romantic" (p.123), it may be that hypotheses such as these which seem impossible to test biologically can be tested computationally.

2.4 "Coevolution" Between Organisms and Their Environments

It is easy to envision evolution as a process by which passive individuals are actively selected (for or against) by their environments. In this view, genetic alternatives are generated and the environment culls some and perpetuates others. A related simplification is to assume that all the information determining the outcome of these evolutionary experiments resides in the individual's genes.

As the discussion of reaction norms above makes clear, a wide range of "epigenetic" variation can be elicited from the same genome by the environment. Waddington (1975) is often credited with going the next step, elevating the environment to a full partner in a coevolutionary process. Certainly environments selects individuals, but through a process sometimes called "niche selection" individuals---especially behaving individuals---also select their environments (a popular song refers to this as "finding a place where the weather suits my clothes"). Waddington (1960) identified this capacity of individuals as a third "exploitative system," operating together with genetics and natural selection.

In The Ontogeny of Information, Oyama (1985) argues that a more balanced view of genetic and environmental roles can also help us move beyond polarized debates of "nature" versus "nurture." Common metaphors for the genes (e.g., programs, blueprints, etc.) as the information-carrying component in evolution all rest on an inappropriate "preformationist" view of information, as if information "...exists before its utilization or expression":

"Instead, it is ontogenesis, the inherently orderly but contingent coming into being, that is essential about the emergence of pattern and form..." (p.3).

"Nativism and empiricism require each other as do warp and weft. What they share is the belief that information can preexist the processes that give rise to it. Yet information `in the genes' or `in the environment' is not biologically relevant until it participates in the phenotypic processes" (p.13).

"[Thus there is] causal symmetry... whereby genetic and nongenetic factors alike can be sources of variation in form and whereby constancy of form generally requires constancy in developmentally relevant aspects of both genome and environment" (p.14).

This theme--causal symmetry between organisms and their environments--is taken up in several of the readings. Schull's exposition of James' work (Chapter 17) makes it clear that this was one of James' central ideas; James extended notions of selection and organism/environment causal symmetry to psychology, social systems, and cultural evolution. This issue is also touched on in Bateson's paper (Chapter 9), with his mention of "extra regulators"---mechanisms by which organisms are able to control their environments and thus keep environmental variables within homeostatic limits. Even Skinner, who might be expected to focus exclusively on environmental characteristics, expresses an even-handed balance between factors [6]:

"Early behaviorists, impressed by the importance of newly discovered environmental variables, found it particularly reinforcing to explain what appeared to be an instinct by showing that it could have been learned, just as ethnologists have found it reinforcing to show that behavior attributed to the environment is still exhibited when environmental variables have been ruled out" (Skinner, this volume, p.274).

In this book, Todd's paper on sexual selection (Chapter 21) considers organism/environment interactions from another angle. Under sexual selection, organisms act as selectors on each other, yielding a coevolutionary process that can run amok as "runaway sexual selection." Finally, Menczer and Belew's paper (Chapter 13) on "Latent Energy Environments" reports on LEE, a computer model specifically designed to investigate organism/environment interactions.

2.5 Are Learning and Evolution Two Faces of Same Coin?

It is quite evident that learning individuals have evolved, and so we have immediate reason to be interested in situations that involve both learning and evolution. A less obvious connection between learning and evolution are arguments that the two forms of adaptation are in fact variations on a single, common theme. If true, our explanations of each will share critical features, for deep reasons.

A typical starting point is Broadbent's (1961) Law of Effect: actions followed by rewards are repeated. This seems to fit both forms of adaptation quite well. Dennett explains "Why the Law of Effect Won't Go Away":

"The Law of Effect and the principle of natural selection are not just analogies; they are designed to work together.... If creatures with some plasticity in their input-output relations were to appear, some of them might have an advantage over even the most sophisticated of their trophistic cousins. Which ones? Those that were able to distinguish good results of plasticity from bad, and preserve the good. The problem of selection reappears and points to its own solution: let some class of events in the organisms be genetically endowed with the capacity to increase the likelihood of the recurrence of behavior-controlling events upon which they act" (Dennett, 1981, p.75).

Over a century earlier, Spencer attempted to make a similar parallel between learning by individuals and evolution by populations. This attempt is reflected here in two readings from Spencer's books (Chapters 15--16). The reader is warned that in support of his arguments, Spencer sometimes expresses alarmingly racist and unscientific views about differences in human intelligence. For example:

"...thus it happens that the European comes to have from twenty to thirty cubic inches more brain than the Papuan. Thus it happens that faculties, as that of music, which scarcely exist in the inferior human races, become congenital in the superior ones. Thus it happens that out of savages unable to count even up to the number of their fingers, and speaking a language containing only nouns and verbs, come at length our Newtons and Shakspeares [sic]" (p.242).

Spencer was certainly not alone. Even William James, acknowledged by many as one of the greatest thinkers of the twentieth century, believed that:

"[the]... absence of prompt tendency in [the male] brain to set into particular modes is the very condition which insures that it shall ultimately become so much more efficient than the woman's.... [T]he masculine brain deals with new and complex matter... in a manner which the feminine method of direct intuition, admirably and rapidly as it performs within its limits, can vainly hope to cope with" (James, 1927/1955, pp.691--692).

Rather than attempting to exorcise such offenses from the historical readings, we encourage the modern student of evolution, behavior, and intelligence to remember the role tacit preconceptions and social agendas have played in the history of these subjects. As publications as recent as The Bell Curve (Herrnstein and Murray, 1994) demonstrate, these dangers remain with us.

Such political issues aside, the parallel between learning and evolution that Spencer identifies takes a more modern form with Karl Popper. His famous phrase, "Let our hypotheses die in our stead," is made quite explicit by models like that of Hinton and Nowlan (Chapter 25). In such models the tradeoff between a population's use of mortal individuals and an individual's use of repeated trials is as direct as can be. A bit of trial-and-error experience in a lifetime is viewed as a "cheap trial" that is easier to test as an individual's hypothesis than going to the bother of constructing an entirely new genetic experiment.

While the tradeoff between a life and a bit of individual learning can be effected in a computer model by simply renaming appropriate variables (e.g., Allele ---> Guess), it is important to remember that in real biological systems the two forms of adaptation rely upon entirely different mechanisms. The exact biochemistry of learning in the nervous system remains an area of intense investigation, but many features---phosphate channels, neurotransmitters, long-term potentiation, etc.--are known. Similarly, many details of population and molecular genetics--nucleotide substitution patterns, polymorphism's, etc.--are also well known. What we do know of the biological processes of learning and evolution suggests that however similar the adaptive problems these processes face, and even if they both do in fact share a common Law of Effect-style learning rule, these similarities exist at what a computer scientist would call the "logical" level only. And just as every logical computer design must be transformed into a physical design, incorporating implementation constraints imposed by features of the hardware, we should expect that the biological phenomena of learning and evolution will also be subject to their respective biological "implementation constraints." To use Broadbent's language, the generation of an "action," the perception of "reward," the "memory" required to allow the same action to be repeated, etc., will involve very different representations and modes of access to these representations in the individual and population cases. While it is provocative to think of learning and evolution as two examples of a similar adaptive technique, their differences are equally provocative.

3. Caveats

Almost as important as the themes these chapters share may be warnings about the problems we have encountered, and the steps we have taken to avoid them. Two require special attention.

The first is a closely related topic we purposefully avoid: cultural evolution. If we use "culture," (or perhaps "proto-culture," Lumsden and Wilson, 1981) to refer to all of the many ways conspecifics affect one another via regularized modifications to their shared environment, the connections with our themes are obvious and many. Culture is a prime determinant of individuals' experience. If the environment shaping and shaped by adaptive populations and individuals is as central as we argue above, then certainly cultural systems which ritualize the environmental experience of individuals across generations must be hugely important ( Hutchins, 1995). If the parallels between learning and evolution just mentioned are striking, those relating processes of cultural change to evolution are no less so. Dawkins (1976) has popularized the notion of cultural "memes" (which may perhaps take the form of myths, hypotheses, theories, books, etc.) that are to be considered as the analog for genes in the evolutionary system; others have advanced very similar arguments (Popper, 1972; Campbell, 1974). In a similar vein, Bateson's article (Chapter 9) draws an intriguing parallel between the Baldwin effect and the actions of a legislative body, "affirming in law that which has already become the custom of the people" (p.117, this volume). Culture certainly seems---and can be modeled as (e.g., see Belew, 1990)---a third, intermediate form of adaptation, operating between learning and evolution. The time scale of the environmental regularities to which culture "attends" (cf. Section 2.3 above) falls squarely between the paces of evolutionary change and individual lifetimes. In all these ways, processes of cultural adaptation seem natural bedfellows with learning and evolution.

Nevertheless, in this book we skirt the issue almost completely, for several reasons. First, as the brief list of issues above and the chapters to follow hopefully illustrate, it seems quite hard enough to consider interactions among individual plasticity and specic evolution without complicating our topic still further. Second, these issues appear prior to those involving culture, and their independent consideration therefore seems justified. In Section 2.3 above, we briefly considered the evolution of the first species with significant learning abilities, in comparison to prior species whose adaptation depended exclusively on evolutionary forces. In the same way, we propose to restrict our attention to those earlier, more "primitive" situations in which learning and evolution interact, without an appreciable role for culture.

Our third reason is that a full analysis of culture's role as yet another, equally powerful adaptive force seems hard indeed. This is quite apparent already from studies of bird song, canine play, etc. The enormous complexity, and danger of facile simplification, that face us as we attempt to address human cultural phenomena is staggering. The fourth and final reason for avoiding culture as much as possible is just how attractive the parallels between cultural adaptation and evolution are. Viewing myths or legislation as a culture's genetic material is clearly provocative. But whether such images are useful merely as metaphors or instead point toward fundamental characteristics of adaptive systems depends on better understanding of the phenomena of adaptation and more concrete data concerning cultural change. For now, then, we must satisfy ourselves with the hope that a better understanding of learning and evolution will prepare us for this next critical stage of investigation.

The second caveat concerns the fine line we must walk between believing in adaptation's power and succumbing to the "adaptationist fallacy." That fallacy, that evolution inexorably improves its designs until reaching perfection, has dogged evolutionary theory for much of its history. Modern evolutionary analysis, certainly since Gould and Lewontin's (1979) devastating critique of the "Panglossian paradigm," has made it clear that the biological process is much more stochastic and much less driven to perfection than Pangloss might desire; being good enough to get by is typically just fine. Our cause for concern about adaptationism, then, is motivated first by the fact that this well-known message of the biologists may not have filtered fully into the other disciplines.

But the issue is complicated further by the participation of computer scientists in this discussion. Typically these scientists work as part of engineering schools, within which the search for good, and preferably optimal, designs is almost axiomatic. Our initial distinction between computational models and algorithms gains real teeth here. To the extent that we are attempting to build veridical models of biological evolution, we should remain wary of adaptationist tendencies toward optimal outcomes. But as engineers of computational algorithms, we will be expected by our peers to achieve just this optimality. It therefore becomes especially important to be clear as to the scientific purpose of any one piece of work.

4. The Benefits of Interdisciplinary Research

As we said above, a distinguishing characteristic of this book---one it shares with much of the work associated with the Santa Fe Institute---is that it depends critically on the shared perspective provided by the involvement of scientists from multiple disciplines. As anyone who has participated in interdisciplinary work can attest, such conversations can often be unproductive and even painful. Beyond our central focus on the interactions between two forms of adaptation, we believe this book can also provide a case study of a successful interdisciplinary interaction. We will mention several characteristics that seem important to that success here.

First, we did not attempt to blur the distinctions among our participants' background disciplines. Said positively, we believe that the best way to be interdisciplinary is to respect disciplines. We relied, for example, on the biologists attending our meeting to tell us about evolution and the environment; they necessarily did this from the paradigmatic perspective of biology, something they had spent their careers absorbing. As organizers and editors, our respect for each of the three disciplinary perspectives of our workshop (and of this book) has caused us to, if anything, artificially highlight the distinctions between them. For example, the readings that follow have been grouped into Biological, Psychological and Computational sections, when in fact almost every reading has at least some aspects of the other two. These groupings are therefore in some cases fairly arbitrary, sometimes reflecting the discipline with which the author is primarily associated (e.g., the chapter by Piaget (Chapter 19) is in the Psychology section, although it largely deals with intersections between biology and psychology), and sometimes reflecting the discipline in which the reading has had the most influence (e.g., Hinton and Nowlan's model of the Baldwin effect is in the Computer Science section, Chapter 25). The unifying views of each of the three areas are captured in the "overview" chapters, summarizing the meetings of the three disciplinary subgroups.

The subtleties of each paradigm's perspective, and their differences as they address a shared set of issues, are impossible to adequately summarize here. Some very interesting telltale signs are provided, however, by quite simple evidence: the words each of these participants use. Logical positivists aside, current philosophy of science acknowledges the tremendous burden carried by the "keywords" of every science. This point has been made especially clear in biology by Fox Keller and Lloyd's (1992) Keywords in Biology:

"[Words] serve as conduits for unacknowledged, unbidden, and often unwelcome traffic between worlds. Words also have memories; they can insinuate theoretical or cultura past into the present. Finally, they have force. Upon examination, their multiple shadows and memories can be seen to perform real conceptual work, in science as in ordinary language" (p.2).

"Indeed, it is precisely because of the large overlap between forms of scientific thought and forms of societal thought that "keywords"---terms whose meanings chronically and insistently traverse the boundaries between ordinary and technical discourse---can serve not simply as indicators of either social meanings and social change or scientific meaning and scientific change, but as indicators of the ongoing traffic between social and scientific meaning and, accordingly, between social and scientific change" (pp.4--5, emphasis in original).

The confusion's caused by "simple" misinterpretation of critical words in our interdisciplinary conversations are therefore far from simple. In this and in other Santa Fe Institute workshops, we have found that one of the hardest parts of interdisciplinary discussions is reaching a consensus on the meaning of terms central to the discussion. In this spirit, we have collected a set of the most troublesome keywords that arose in a glossary at the end of this volume. As with the Keywords book, these characterizations of the words' meanings are:

"...not intended to provide definitive or correct definitions...[but] to provide a rough map of some of the territory of dispute and change" (Keywords, p.6).

Unlike Keywords, however, we have restricted our attempts to paragraphs rather than multiple pages. The "definitions" have come from our authors and workshop participants, a group of students at the University of California at San Diego who were early readers of this manuscript, and the dictionary. In some cases this has resulted in multiple definitions, reflecting some of the terminological debate that arose in our meeting.

In many ways, this glossary forms a fitting conclusion for our volume. It reflects the variety of perspectives of our interdisciplinary group, and captures some of the dynamic of the meeting from which this printed record has sprung. It highlights the role artifacts of the scientific process, from words to computational models, play in shaping the science they convey. And while these words' characterizations, and the rest of this book's elements, are far from definitive, we hope that they may put subsequent discussion of these important issues on firmer ground.

References

Agre, P. E. "Computational Research on Interaction and Agency." Artificial Intelligence 72 (1995): 1-52.
[return to text]

Belew, R. K. "Evolution, Learning, and Culture: Computational Metaphors for Adaptive Search." Complex Systems 4(1) (1990): 11-49.
[return to text]

Brauth, S. E., W. S. Hall, and R. J. Dooling, eds. Plasticity of Development. Cambridge, MA: MIT Press, 1991.
[return to text]

Broadbent, D. E. Behavior. New York: Basic Books, 1961.
[return to text]

Cairns Smith, J., J. Overbaugh, and S. Miller. "The Origins of Mutants." Nature 335 (1988): 142-145.
[return to text]

Campbell, D. T. "Evolutionary Epistemology." In The Philosophy of Karl Popper, edited by P. A. Schlipp. LaSalle, IL: Open Court, 1974.
[return to text]

Cecconi F., F. Menczer, and R. K. Belew. "Maturation and the Evolution of Imitative Learning in Artificial Organisms." Adaptive Behavior 4(1) (1996): in press.
[return to text]

Dawkins, R. The Selfish Gene. New York: Oxford University Press, 1976.
[return to text]

Dennett, D. C. "Why the Law of Effect Won't Go Away." In Brainstorms. Cambridge, MA: MIT Press, 1981.
[return to text]

Elman, J. L., E. A. Bates, M. H. Johnson, A. Karmiloff-Smith, D. Parisi, and K. Plunkett. Rethinking Innateness: A Connectionist Perspective on Development. Cambridge, MA: MIT Press (in press).
[return to text]

Foster, P. L. "Adaptive Mutation: The Uses of Adversity." Annual Review of Microbiology 47 (1993): 467-504.
[return to text]

Fox Keller, E., and E. A. Lloyd, eds. Keywords in Evolutionary Biology. Cambridge, MA: Harvard University Press, 1992.
[return to text]

Gordon, D. M. "Phenotypic Plasticity." In Keywords in Evolutionary Biology, edited by E. Fox Keller and E. A. Lloyd. Cambridge, MA: Harvard University Press, 1992.
[return to text]

Gould, S. J., and R. C. Lewontin. "The Spandrels of San Marco and the Panglossian Paradigm: A Critique of the Adaptationist Programme." Proceedings of the Royal Society of London B205 (1979): 581-598.
[return to text]

Herrnstein, R. J., and C. Murray. The Bell Curve: Intelligence and Class Structure In American Life. New York: Free Press, 1994.
[return to text]

Hutchins, E. Cognition in the Wild. Cambridge, MA: MIT Press, 1995.
[return to text]

James, W. The Principles of Psychology. Great Books of the Western World (vol. 53), Encyclopedia Brittanica, Inc., (1927/1955). (First published in 1927 by H. Holt and company, New York.)
[return to text]

Lorenz, K. Behind the Mirror: A Search for the Natural History of Knowledge. (R. Taylor, translator). New York: Harvest/HBJ, 1973.
[return to text]

Lumsden, C. J., and E. O. Wilson. Genes, Minds, and Culture: The Coevolutionary Process. Cambridge, MA: Harvard University Press, 1981.
[return to text]

Mayr, E. The Growth of Biological Thought. Cambridge, MA: Harvard University Press, 1982.
[return to text]

Oyama, S. The Ontogeny of Information: Developmental Systems and Evolution. Cambridge: Cambridge University Press, 1985.
[return to text]

Piaget, J. Behavior and Evolution. New York: Random House, 1978.
[return to text]

Plotkin, H. C. "Learning and Evolution." In The Role of Behavior in Evolution, edited by H. C. Plotkin. Cambridge MA: MIT Press, 1988.
[return to text]

Popper, K. R. Objective Knowledge: An Evolutionary Approach. Oxford: Clarendon Press, 1972.
[return to text]

Shapiro, J. A. "Adaptive Mutation: Who's Really in the Garden?" Science 268 (1995): 373--374.
[return to text]

Simon, H. A. "Cognitive Science: The Newest Science of the Artificial. Cognitive Science 4 (1980): 33--46.
[return to text]

Suchman, L. A. Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge, MA: Cambridge University Press, 1987.
[return to text]

Waddington, C. H. "Evolutionary Adaptation." In Evolution of Life, edited by S. Tax. Chicago, IL: University of Chicago Press, 1960.
[return to text]

Waddington, C. H. Evolution of an Evolutionist. Ithaca, NY: Cornell University Press, 1975.
[return to text]

Weismann, A. The Germ-Plasm: A Theory of Heredity. New York: Scribners, 1893.
[return to text]

Wright, S. "Genic and Organismic Selection." Evolution 34 (1980): 825-843.
[return to text]

Footnotes

[1] There are two minor variations on the basic Preface + Classic pattern. In Chapter 3, J. Schull has provided a more holistic introduction to the classics of Lamarck, Baldwin, Lloyd Morgan, Waddington, Simpson, and Bateson (Chapters 4, 5, 6, 7, 8, and 9, respectively). Also, since the relevant works of William James fall across a number of his writings, Schull has intermixed extended quotations from James with his own commentary on these in Chapter 17. [return to text]

[2] Other issues surrounding interdisciplinary research are considered below in Section 4. [return to text]

[3] The word "plastic" was in fact the defining word of our workshop: Plastic individuals in evolving populations." Among this group of self-selected psychologists, computer scientists, and biologists the appropriate sense of "plastic" was understood. Our publishers have convinced us, however, that the typical book buyer is more likely to be reminded of recycling and perhaps "The Graduate," not cognition! [return to text]

[4] Note that Bateson (Chapter 9) makes this blockage his very first premise of theory building. [return to text]

[5] Of course, we must be careful as human observers not to impose our chauvinistic notion of generational time onto this adaptation; evolution can be a fairly rapid process in some species. The important comparison is between the relative rates of evolutionary and individual change. [return to text]

[6] Though note the bias expressed by his invocation of the languate of learning, "...found it particularly reinforcing," as an explanation for the behaviors of the two classes of scientists! [return to text]

Adaptive Individuals in Evolving PopulationsModels and Algorithms