Neural Correlates of Infants’ Visual Responsiveness to Facial Expressions of Emotion

CHARLES A. NELSON MICHELLE DE HAAN
University of Minnesota Institute of Child Development Minneapolis, Minnesota

An extensive literature documents the infant’s ability to recognize and discriminate a variety of facial expressions of emotion. However, little is known about the neural bases of this ability. To examine the neural processes that may underlie infants’ responses to facial expressions, we recorded event-related potentials (ERPs) while 7-month-olds watched pictures of a happy face and a fearful face (Experiment 1) or an angry face and a fearful face (Experiment 2). In both experiments an early positive component, a middlelatency negative component and a later positive component were elicited. However, only when the infants saw the happy and fearful faces did the components differ for the two expressions. These results are discussed in the context of the neurobiological processes involved in perceiving facial expressions. 0 1996 John Wiley & sons. Inc.
Keywords: ,fuciul expression; emotion; electrophysiologicul; development; recognition
The human face plays an extraordinarily important role in social communication between infants and adults prior to the onset of language. The emotions expressed in a face may be particularly important because they represent the most prominent channel for perceiving affect (although the voice also conveys affect; see Fernald, 1993;WalkerAndrews & Lennon, 1991). It is not surprising, then, that numerous investigators over the past 15 years have attempted to determine how well infants can recognize different facial expressions of emotion (for reviews see Nelson, 1985, 1987; Nelson & de Haan, in press; Oster, Daily, & Goldenthal, 1989; Walker-Andrews, 1988).
The focus of much of the work in this area has been to determine whether infants can (a) discriminate among different types of facial expressions and (b) recognize that different examples of one type of expression belong to the same category. For example, to test for discrimination, 3-month-olds in one study were habituated to a smiling face or a frowning face and then tested with both expressions (Barrera & Maurer, 1981).

Reprint requests should be sent to Charles A. Nelson, Institute of Child Development, University of Minnesota, 51 East River Road, Minneapolis, MN 55455. U.S.A.
Received for publication 21 March 1995 Revised for publication 1 I September 1995 Accepted for publication 26 June 1996

Deuelopmenrul Psychobiology 29(7):577-595 (1996) 0 1996 by John Wiley & Sons, Inc.

CCC 0012-1630/96/070577-19

578 NELSON AND DE HAAN
Infants looked longer at the novel expression than at the one to which they had habituated. The results of this and other studies (e.g., Schwartz, Izard, & Ansul, 1985;YoungeBrowne, Rosenfeld, & Horowitz, 1977) suggest that infants only a few months old can discriminate among different types of emotional expressions. To test for categorization, 7-month-olds in one study were first familiarized to three different women posing a happy expression and were then tested with a new model posing a happy expression and a fearful expression (Nelson, Morse, & Leavitt, 1979). Infants showed recovery of looking to the fearful expression but not to the happy one, suggesting that they recognized the similarity between the happy expressions posed by the new and old models. The results of this and other studies (Caron, Caron, &Myers, 1982;Kestenbaum & Nelson, 1990; Ludemann & Nelson, 1988; Nelson & Dolgin, 1985; Nelson et al., 1979) suggest that by 6 to 7 months of age, infants recognize that different examples of the same expression belong to the same category.
One effect that was observed in several of these studies is that the order in which infants are exposed to different expressions influences whether or not they show evidence of discrimination and categorization. For example, in one study (Schwartz et al., 1985) 5-month-olds showed recovery of looking to fearful, sad, or interest expressions after habituation to an angry one, but did not show recovery of looking to an angry expression after habituation to fearful, sad, or interest ones. In another study (Nelson et al., 1979),7-month-olds showed recovery of looking to a fearful expression following familiarization to one or more examples of happy, but they did not show recovery of looking to a happy expression following habituation to one or more examples of fear. These effects of order of presentation are intriguing because they suggest that exposure to an expression during habituation may not be the only factor influencing infants’ responses to that expression.
One method that can be used to study reactions to expressions that are not related to recent experience during experimental familiarization or habituation is the visual preference method. In this method, infants are typically shown two expressions and their distribution of looking between the two is noted. Because this method does not require infants to remember one stimulus and discriminate it from another, it may provide the best reflection of infants’ spontaneous responses to facial expressions. For example, 7-month-olds looked longer at fearful expressions than at happy expressions (Nelson & Dolgin, 1985), 4-and 6-month-olds look longer at a joyful expression than at an angry or a neutral one (LaBarbera, Izard, Vietze, & Parisi, 1976),and 5-montholds look longer at a film of a smiling face than at a film of a neutral or frowning face (Wilcox & Clayton, 1968).
Although these results support the idea that infants’ responses to expressions may not be influenced only by recent exposure during habituation, the results do not resolve the question of why infants look longer at certain expressions than at others. One factor that may influence infants’ responses is their experience with expressions in the natural environment. For example, infants may see happy expressions more frequently than fearful expressions. If fearful expressions are less familiar than happy ones, they may be more difficult to habituate to and may still seem “novel” relative to happy expressions in tests of recovery of looking (Nelson et al., 1979).Another factor that may contribute to infants’ responses is the social signal value of the stimulus. For example, fearful expressions may induce a “defensive response” in infants associated with increased attention to the fearful face (Nelson et al., 1979). This obligatory reaction would lead infants to continue to look longer at the fearful expression than at a happy one even after habituation to the fearful expression. Similarly, infants may not show recovery

FACIAL EXPRESSIONS O F EMOTION 579
of looking to angry expressions following habituation to other expressions because angry expressions induce gaze aversion (Schwartz et al., 1985).
Overall, the results of investigations of infants’ recognition of expressions thus suggest that: (a) infants can discriminate among and categorize at least some expressions early in life, but (b) their reactions to expressions may not be only the result of experience during an experiment. These observations have led a number of authors to propose, following Darwin (18721, that the recognition of facial expressions may represent a selective adaptation that has resulted in the creation of specialized neural systems that subserve this ability (e.g., Charlesworth & Kreutzer, 1973; Nelson, 1987, 1993). Although the observation that infants can categorize expressions early in life does not necessarily prove that there is a specialized neural system for recognizing expressions (an alternative possibility is that there is a more general system that emerges early in life and mediates categorization of several different types of pattern), recognizing expressions could be adaptive for the infant if, for example, recognizing a stranger’s angry expression facilitates a crying response which in turn brings the caretaker to defend the child (see Bowlby, 1969 for discussion). Because human infants just a few months old are able to discriminate among different types of expressions, the neural systems that mediate recognition of expressions may require relatively little postnatal experience in order to develop (Although it is also possible that experience is necessary to facilitate the development of the ability to evaluate or interpret facial expressions.)
It is difficult to provide direct support for this hypothesis (for more detailed discussion see Nelson & de Haan, in press) partly because many of the methods used to study the neural bases of behavior are impossible (e.g., single-cell recordings o r experimental lesions) or inappropriate (e.g., positron emission tomography, or PET) for use with healthy human infants. One method that can be used to study brain activity in healthy human infants is to record event-related potentials (ERPs). ERPs are transient changes in brain activity that occur in response to a discrete event (e.g., a brief presentation of a picture of expression). By measuring the brain responses that occur during repeated presentations of stimuli (e.g., a 500-ms stimulus presented 100 times), reliable patterns of brain activity that reflect the neural and cognitive processes involved in responding to these stimuli can be revealed. In previous research, this method has been used successfully to study memory in infants (for review see Nelson, 1994), and emotion recognition in children (e.g., Nelson & Nugent, 1990) and adults (e.g., Lang, Nelson, & Collins, 1990).
In the work reported in this article, ERPs were used to investigate whether there are neural systems within the human infant’s brain that are activated in a specific way when infants look at facial expressions. We used a method similar to the visual preference method, in which infants are shown expressions without prior familiarization. Each expression was presented with equal probability so that exposure during the experiment was the same for each expression. This method may be most sensitive to the brain’s obligatory responses to facial expressions because it does not require infants to remember one stimulus and discriminate it from another (i.e., habituate the infant to one expression and test for recognition by contrasting the familiar expression with a novel expression). We predicted that the patterns of brain activity invoked by different facial expressions should differ from one another if (a) separate neural systems are involved in the recognition of different expressions or (b) a single neural system is activated in different ways when different expressions are recognized. Although in neither case would we “prove” the existence of neural tissue specialized for responding

580 NELSON AND DE HAAN
to facial expressions, we would, nevertheless, have evidence that points in that direction. A similar approach has been used with adults to compare the neural processes involved in perceiving faces and other objects (e.g., Bentin, Allison, Puce, Perez, & McCarthy, 1995).
In Experiment 1 , we tested infants with a happy expression and a fearful expression. Behavioral studies of infants’ visual preferences show that infants look longer at fearful expressions than at happy ones (e.g., Nelson & Dolgin, 1985). Fearful expressions may recruit more attention than happy ones because they are more novel and/or because their social signal value elicits enhanced attention. In previous ERP studies of infants’ visual recognition memory, it has been demonstrated that stimulithat presumably recruit more attention invoke a greater negative component (see Nelson, 1994 for further discussion of the functional significance of infant ERP components). Based on these results, we predicted that a negative component (latency approximately 400-800 ms) in the ERP waveform would be larger for the fearful expression than for the happy expression. In the studies of infants’ visual recognition memory, brain activity returned to baseline following this negative component for stimuli that were fully encoded (highly recognizable) but showed a positive slow wave for stimuli that were only partially encoded (Nelson & Collins, 1991). We predicted that (a) if 7-month-olds have relatively little experience with fearful faces in their natural environment and (b) during the experiment too few trials of fear were presented to allow full encoding and recognition, then infants should show a more prominent late-positive component for the fearful face than the happy one. In contrast, the response to the happy face should take the form of a return to baseline.
In Experiment 2, we tested infants with an angry and a fearful expression to determine whether their ERP responses would differ for two negative expressions. The results of behavioral studies suggest that, by 4 to 5 months, infants are able to discriminate between these expressions (Schwartz et al., 1985; Serrano, Iglesias, & Loeches, 1992).However, there are no studies of infants’ visual preferences for these two expressions to show how infants would respond without prior familiarization. If infants are equally unfamiliar with fearful and angry expressions and/or they are perceived as the same social signal (e.g., both perceived as “negative”), then the morphology and topography of the ERPs should be similar for both expressions. Alternatively, if angry expressions and fearful expressions are recognized as different social signals, then infants’ ERPs may differ to the two expressions. Unfortunately, it is not possible to specify precisely how this would be reflected in the ERPs.
General Methods
Electroencephalogram (EEG) Recording
The EEG was recording using silver-silver-chlorided (AgAg-C1) electrodes placed over midline (02, Pz, Cz, Fz) and lateral scalp (T3 and T4), according to the International 10-20 system (Jasper, 1958), and referenced to linked ears. A ground electrode was placed on the forehead. The electrodes were held in place with foam pads, Grass EC2 cream, and cloth headbands. Impedances were accepted if they were less than 10 kohms, aithough they were generally less than 5 kohrns.
The electrooculogram (EOG) was recorded from bipolar miniature electrodes placed vertically above and below one eye bisecting the midline. The eye electrodes were held in place with small adhesive collars.

FACIAL EXPRESSIONS OF EMOTION 581
All bioelectrical signals were recorded using a Grass Neurodata Acquisition System with Model 12A5 amplifiers. EEG gain was set to 20,000 and EOG gain was set to 5,000. The bandpass was 0.1 to 30 Hz, with a 60-Hz notch filter engaged.
Procedure
Each infant was tested individually while sitting on the parent’s lap facing a slide screen approximately 40 cm away. The slide screen was surrounded by black panels that blocked the infant’s view of the room behind the screen and to hidher sides. There were small peepholes in the panel through which observers could watch the infants.
The session began when the infant fixated on the screen. Each trial consisted of a 100-msbaseline, followed by a 500-mspresentation of the stimulus, followed by 1200 ms during which the screen was blank. EEG was sampled every 10 ms (100 Hz) throughout the trial (i.e.. 100 ms pre-event through 1700ms after event onset). The intertrial interval varied randomly between 500-1200 ms, and during this time the screen was blank.
An observer behind the screen watched the infant through the peephole and, if the infant looked away, signaled the computer via push button to present that trial again (Note that infants’general direction of gaze was monitored, not their corneal reflections.) If needed, the observer tapped the screen or shook a rattle to attract the infant’s attention to the screen. Brain activity from the trials during which the infant was not looking at the screen or during which the observer attempted to attract the infant’s attention was not recorded. The session continued until the infant had seen the maximum number of trials (70) or became too fussy or bored to attend. The mean number of trials presented to each subject was 64 +- 9.0 in Experiment 1 and 64 & 11.5 in Experiment 2.
ERP Data Reduction
Data were digitized online and then stored on removable hard-disk (Bernoulli) cartridges. After the data were digitized, they were edited for artifacts by a computer algorithm. Any trial in which the EOG signal exceeded 250 microvolts or the EEG signal exceeded A-D values in any 50 ms window was identified by the computer algorithm as containing an artifact. Trials containing EEG artifacts were rejected from the sample, whereas trials containing EOG artifacts were corrected for the influence of eye movement on the EEG (Gratton, Coles, & Donchin, 1983).Briefly, the correction algorithm employed compares deflections in the EOG with deflections in the EEG, computes a model of the influence of eye movement on the EEG, and based on this model subtracts activity from the EEG that is due to eye-movement artifact.
Individual subject averages were then computed for each stimulus type. Individuals who did not have a minimum of five artifact-free trials for each electrode location for each average were excluded from further analysis. To equalize the signal-to-noise ratio among stimulus conditions, an equal number of trials was randomly selected from the available artifact-free trials in each condition. For example, if there were 15 artifactfree trials for the happy face and 20 for the fearful face, all 15 “happy” trials would be used to compute the average response to the happy face and 15 of the 20 “fearful” trials would be randomly selected to compute the average response to the fearful face, Experiment 1: M = 17.0 trials per average, SD = 5.3; Experiment 2: M = 15.4 trials per average, SD = 6.4. A grand average was then computed for each stimulus.
On the basis of inspection of each of the individual subjects’ averages and the grand averages, time windows containing ERP components of interest were identified. One set

582 NELSON AND D E HAAN
of windows was identified for Oz (140-260 ms, 260-620 ms, 620-720 ms, 720-1060 ms, 1060-1700 ms), one for Pz, Cz, and Fz (140-370 ms, 370-680 ms, 680-1280 ms), and one for T3 and T4 (140-260 ms, 260-680 ms, 680-1700 ms). Peak amplitudes relative to the 100-ms baseline and latency to peaks were determined for ERP activity within each window.
Statistical Analyses
Repeated measures analyses of variance (ANOVAs) and t tests were performed on the ERP peak amplitude and peak-latency measures using the BMDP programs 2V and 3D. Main effects were analyzed using Newman-Keuls tests for post-hoc comparison of means. All p values were corrected with the Greenhouse-Geisser procedure.
Experiment 1 Experiment 1 was designed to investigate whether infants’ brain activity is influ-
enced by the type of emotion expressed in a face. We tested infants with a happy expression and a fearful expression because previous studies measuring how long infants look at these two expressions suggest that, even without prior familiarization to either expression, infants look longer at fearful expressions than at happy ones (Nelson & Dolgin, 1985).
Method
Participants The final group of participants were nineteen 7-month-old infants ( M = 213 days, range 209-222 days; 9 males). An additional 23 infants ( M = 215 days, range 206-225 days; I 1 males) were tested but were not included because of eye and/or body movements that resulted in EEG or EOG artifacts ( n = 12), because of fussiness or inattention ( n = 8), because of equipment failure ( n = l), or because of a procedural error ( n = 2).
Stimuli The stimuli were two black-and-white slides (taken from the Ekman collection; see Ekman & Friesen, 1976) of one woman posing a happy expression (Slide 9) and a fearful expression (Slide 7). The two expressions were presented randomly with equal probability.
Results
For each ERP component, a separate repeated measures ANOVA with emotion (happy, fearful) and lead (Pz, Cz, Fz, T3, T4) as the within subject factors was computed

FACIAL EXPRESSIONS OF EMOTION 583
Fig. I . ERP grand averages for happy (solid line) and fearful (dashed line) faces in Experiment I ( n = 19). The stimulus was presented from 0-500 ms. Each waveform was shifted so that baseline (i.e., activity from - 100 to 0 ms; not shown) amplitude was equivalent across the different leads and conditions. The center plots represent the midline leads moving in an anterior (Fz) to posterior ( 0 2 )direction. The left (T3) and right (T4) temporal leads are displayed on the outside.
for peak latency and peak amplitude. Lead Oz was analyzed separately using r tests because the morphology of components differed from other midline leads. However, because none of the analyses for Oz yielded significant effects, these results will not be discussed further.
The grand averages for each expression at each lead are illustrated in Figure 1 and the peak amplitudes and peak latencies are listed in Tables 1 and 2 respectively. Note that the maximum and minimum amplitudes listed in the tables do not correspond exactly to the maximum and minimum values in the graphed grand averages. This is because the values are computed in slightly different ways. The amplitudes listed in the table (and used in the analyses) are computed by identifying the peak amplitude anywhere within a particular time window for each subject's average, and then taking

584 NELSON AND DE HAAN

Table 1 Happy-Feurful Expression: Peak Amplitudes in Microvolts

a. E d v Positive Component ( n = 19)

Lead

Expression

Pz CZ Fz

T3

T4

Happy mean (SD) 26.84 (22.28) 21.58 (17.80) 22.47 (17.93) 15.11 (22.13) 16.89 (17.03) Fearful mean ( S D ) 16.79 (15.02) 12.53 (15.54) 5.00 (17.40) 6.26 (16.40) 3.26 (15.61)

h. Negative Component ( n = 19)

Expression

Lead
PZ cz Fz T3 T4

Happy mean (SD) -6.05 (21.92) - 14.37 (19.03) - 13.11 (25.79) - 16.05 (35.86) - 14.89 (37.91) Fearful mean (SD) - 17.74 (21.67) -21.68 (14.00) -26.58 (25.37) -27.00 (24.98) 34.79 (38.88)
-~~
c. Late Positive Component ( n = 19)

Expression
Happy mean (SD) 33.21 (21.82) Fearful mean (SD) 23.56 (26.81)

33.16 (24.82) 18.63 (25.43)

Lead
28.32 (31.80) 7.26 (27.65)

30.10 (31.91) 18.58 (38.07)

43.84 (38.91) 19.63 (39.75)

the average of these values across the group. In contrast, the graphed grand averages are computed by calculating the average amplitude across subjects at each time point. Variability between subject in the latency of peak amplitudes will cause the peaks in the graphed grand averages to be “flattened” (i.e., the peak may be broader and of lower amplitude) compared to the values listed in the table.
Early Positive Component
Amplitude There was a main effect of emotion on the amplitude of this component, F( 1,18) = 4.89, p < 0.05. Inspection of the means showed that this occurred because the component invoked by the happy face had a larger amplitude than the component invoked by the fearful face (see Figure 1 and Table 1). In addition, there was a main effect of lead, F(2,36) = 7.16, p < 0.01. Follow-up tests showed that the component was larger at Pz than at T3 or T4, p < 0.05. There was no Lead x Emotion interaction.
Latency There was a main effect of lead on the latency of this component, F(4,72) = 15.82, p < 0.01. Follow-up tests showed that the maximum amplitude of this component

FACIAL EXPRESSIONS OF EMOTION 585
occurred earlier at lateral leads than at midline leads ( p < 0.05 for all comparisons between midline and lateral leads except Fz with T3, which was not significant; see Table 2). There was no main effect of emotion or Lead x Emotion interaction.

Negative Component
Amplitude
There was a main effect of emotion on the amplitude of this component, F( I , 18) = 6.58, p < 0.05. Inspection of the means showed that this occurred because the component invoked by the fearful face had a larger amplitude than the component invoked by the happy face (see Figure 1 and Table 1). There was no main effect of lead or Lead x Emotion interaction.

Latency
There was a main effect of lead on the latency of this component, F(4,72) = 2.89, p < 0.05. Inspection of the means suggested that the maximum amplitude of this component occurred earlier at lateral leads than at midline leads (see Table 2); however these comparisons were not significant. There was no main effect of emotion or Lead x Emotion interaction.

Table 2 Happy-Fearful Expression: Peak Latencies in Milliseconds
a. Early Positive Component (n = 19)
Lead

Happy mean (SD) 266.31 (64.14) 245.26 (49.03) 240.53 (55.12) 196.84 (26.68) 201.58 (39.90) Fearful mean (SD)276.84 (68.40) 257.89 (48.02) 274.21 (67.52) 204.21 (36.10) 192.11 (41.58)

b. Negative Component ( n = 19)
~~~~

Expression

PZ

cz

Lead Fz

T3

7-4

Happy mean (SD) 558.95 (101.65) 531.05 (101.21) 552.63 (95.44) 500.53 (108.45) 500.00 (120.83) Fearful mean (SD) 564.74 (100.08) 514.74 (86.43) 569.47 (89.90) 523.16 (139.05) 488.95 (128.96)

c . Late Positive Component ( n = 19)

Expression

PZ CZ

Lead Fz

T3

T4

Happy mean (SD) 945.26 (123.93) 982.63 (145.37) 1062.11 (130.32) 1179.47 (227.63) 1296.84 (227.50) Fearful mean (SD)942.11 (151.39) 975.79 (165.88) 1001.05 (174.42) 1183.16 (319.71) 1202.11 (288.49)

586 NELSON AND DE HAAN
Late Positive Component
Amplitude
There was a main effect of emotion on the amplitude of this component, F(1,18) = 4.91, p < 0.05. Inspection of the means showed that this occurred because the component invoked by the happy face had a larger amplitude than the component invoked by the fearful face (see Figure 1 and Table 1 ) . There was no main effect of lead or Lead x Emotion interaction.
Latency
There was a main effect of lead on the latency of this component, F(4,72) = 14.32, p < 0.01. Follow-up tests showed that the maximum amplitude of this component occurred earlier at midline leads than at lateral leads ( p s < 0.05 for all comparisons between midline and lateral leads except Fz with T3; see Table 2). There was no main effect of emotion or Lead x Emotion interaction.
Discussion
Both the happy expression and the fearful expression invoked three ERP components: (a) an early positive component, (b) a middle-latency negative component, and (c) a later positive component. The amplitude of the two positive components was greater for the happy expression than for the fearful expression. In contrast, the amplitude of the middle-latency negative component was greater for the fearful expression than for the happy one.
The negative component invoked by both expressions is similar in latency and scalp distribution to a negative component observed in previous studies (Courchesne, Ganz, & Norcia, 1981; Karrer & Ackles, 1987;Nelson & Collins, 1991, 1992;Nelson & Salapatek, 1986).In those studies, this component was thought to reflect infants’ allocation of attention, with a larger amplitude reflecting allocation of more attention (for discussion see Nelson, 1994). In this experiment, the amplitude of the negative component was larger for the fearful face than for the happy face. This suggests that infants allocated more attention to the fearful face, an interpretation consistent with behavioral studies showing that infants look longer at fearful expressions than at happy expressions (Nelson & Dolgin, 1985).
One alternative interpretation is that the negative component itself does not actually differ for the two expressions. Instead, the difference may just be a consequence of the greater positivity for the happy face earlier in the trial (see Figure 1). One way to test this idea is to determine whether the difference between the peak amplitude of the early positive component and the peak amplitude of the negative component (i.e., the peak-to-peak distance in microvolts) differs for the two expressions. If the difference in the negative component is actually a consequence of differences in the earlier positivity, then the peak-to-peak distance should not differ for happy and fearful faces. In contrast, if the difference in the negative component is not only a consequence of the earlier positivity to happy, then the peak-to-peak distance should be greafer for the fearful faces than for the happy

FACIAL EXPRESSIONS OF EMOTION 587
faces. An analysis of the peak-to-peak difference between the early positive and later negative components supports the second interpretation. There was a main effect of emotion, F(1,18) = 4.38, p = 0.05, which occurred because the peak-topeak distance was greater for the fearful faces than for the happy faces. In addition,
there was a Lead x Emotion interaction, F(1,18) = 2.98, p < 0.05, which occurred
because the effect of emotion was most prominent at lateral leads. These results suggest that the differences between ERPs elicited by happy and fearful faces during the middle-latency time window occurred because the fearful face invoked a larger negative component than the happy face.
We predicted that following the middle-latency negative component, the responses to the happy expression would show a return to baseline and the response to the fearful expression would show a positive component. In contrast to this prediction, we found that the response to both faces showed a positive component, and that this positive component was much larger for the happy expression. One explanation is that infants were unable to fully encode even the happy face because the stimuli were presented very briefly (500 ms), and thus the late positive component reflects a memory updating process. Alternatively, this positive component may not be the same component we have observed in our memory studies (e.g., Nelson & Collins, 1991), and thus may reflect a different cognitive operation (i.e., it may not reflect the memory updating of a partially encoded stimulus).
The presence of the early positive component was not predicted. One possibility is that it, like the middle-latency negative component, reflects processes involved in allocation of attention. However, we have not observed the early positive component in previous studies in which similarly aged infants saw neutral faces that were familiar or unfamiliar (de Haan & Nelson, in press; Nelson & Collins, 1991;Nelson & Salapatek, 1986). In contrast, the middle-latency negative component was invoked under those conditions. This raises the possibility that the cognitive processes reflected by this component may be related to the processing of facial expressions but not facial identity. In contrast, the middle-latency negative component may reflect cognitive processes that are involved more generally in visual attention.
There were no Lead x Emotion interactions for peak amplitudes or peak latencies for any of the three components. The absence of Lead X Emotion interactions may suggest that a similar neural system was activated, though possibly to a different extent, by the two expressions. However, this conclusion can only be tentative because the absence of any Lead x Emotion interactions may be a consequence of the relatively small number of electrodes used in the study. In addition, even the presence of Lead X Emotion interactions would not necessarily imply that different neural systems were activated. Condition x Lead interactions can occur even if the same neural system is activated, but to a differing extent, during the different conditions (see McCarthy & Wood, 1985 for a discussion of interpreting Condition X Lead interactions).
Experiment 2
In Experiment 1 , we found that infants’ brain activity distinguished between happy and fearful expressions of emotion. Experiment 2 was designed to determine whether infants’ brain activity would also distinguish between two negative expressions and to determine whether the early positive component would also be invoked by pairs of emotional expressions other than happy and fear. Thus, in Experiment 2 we recorded infants’ brain activity as they watched slides of an angry and a fearful expression.

588 NELSON AND DE HAAN
Method
Participants The final group of participants were nineteen 7-month-old infants ( M = 217 days,
range 210-221 days; 7 males). An additional 34 infants ( M = 214 days, range 205-219 days; 12 males) were tested but were not included because of eye and/or body movements that resulted in EEG or EOG artifacts ( n = 23), because of fussiness or inattention ( n = 4) because of equipment failure ( n = 4), or because of a procedural error ( n = 1).
Stimuli
The stimuli were two black-and-white slides (taken from the Ekman collection; see Ekman 8z Friesen, 1976) of one woman posing an angry expression (Slide 69) and a fearful expression (Slide 68). The two expressions were presented with equal probability.
Results
For each ERP component, a separate repeated measures ANOVA with emotion (happy, fearful) and lead (Pz, Cz, Fz, T3, T4) as the within subject factors was computed for each dependent measure. Lead Oz was analyzed separately using t tests because the morphology of components differed from other midline leads. However, because none of the analyses for Oz yielded significant effects, these results will not be discussed further.
The grand averages for each expression at each lead are illustrated in Figure 2. Peak amplitudes are listed in Table 3 and peak latencies are listed in Table 4.
Early Positive Component
Amplitude
There was a main effect of lead on the amplitude of this component, F(4,72) =2.93, p < 0.05. Inspection of the means suggested that the maximum amplitude was greater at midline leads than at lateral leads (see Table 3), however none of the comparisons were significant. There was no main effect of emotion or Emotion X Lead interaction.
Latency
There was a main effect of lead on the latency of this component, F(4,72) = 28.65, p < 0.01. Follow-up tests showed that the maximum amplitude occurred earlier at lateral leads than at midline leads (all comparisons between midline and lateral leads p < 0.01 except Fz with T3; see Figure 2 and Table 4). There was no main effect of emotion or Lead x Emotion interaction.
Negative Component
Amplitude
There were no effects of emotion or lead on the amplitude of this component.

FACIAL EXPRESSIONS OF EMOTION 589

cz
$5 I

JRgM/
T4

8 &.",$ I f f
Fig. 2. ERP grand averages for angry (solid line) and fearful (dashed line) faced in Experiment 2 (n = 19). The stimulus was presented from 0-500 ms. Each waveform was shifted so that baseline (i.e., activity from - 100 to 0 ms; not shown) amplitude was equivalent across the different leads and conditions. The center plots represent the midline leads moving in an anterior (Fz) to posterior (Oz) direction. The left (T3) and right (T4) temporal leads are displayed on the outside.
Latency There was a main effect of lead on the latency of this component, F(4,72) = 13.26, p < 0.01. Follow-up tests showed that the minimum latency of this component occurred earlier at lateral leads than at midline leads (all comparisons between midline and lateral leads p < 0.01; see Figure 2 and Table 4). There was no main effect of emotion or Lead x Emotion interaction.
Late Positive Component
Amplitude There were no effects of emotion or lead on the amplitude of this component.

590 NELSON AND DE HAAN

Table 3 Angry-Fearful Expression: Peak Amplitudes in Microvolts

a . Early Positive Component ( n = 19)

Expression

PZ

CZ

Lead FZ

Angry mean (SD) 17.94 (18.21) 11.16 (17.71) 14.47 (18.64) Fearful mean (SD) 28.00 (28.46) 19.63 (17.83) 18.26 (17.17)

T3
8.00 (14.82) 16.00 (19.47)

T4
11.95 (18.53) 14.89 (19.68)

b. Negative Component (n = 19)

Expression

~~ ~
PZ

cz

Lead FZ

T3

T4

Angry mean (SD) - 11.95 (27.48) -20.21 (23.47) - 15.37 (25.88) -21.52 (22.29) - 13.05 (21.39) Fearful mean (SD) -9.84 (42.72) - 19.93 (23.73) - 18.73 (23.88) - 11.68 (23.98) - 10.68 (28.13)

c . Late Positiue Componenf (n = 19)

Expression

PZ

Angry mean (SD) 29.21 (30.39) Fearful mean (SD) 30.95 (50.22)

CZ
27.37 (22.46) 27.47 (26.81)

Lead Fz
25.68 (30.02) 25.32 (29.65)

T3
22.00 (22.32) 35.58 (34.52)

T4
31.15 (26.37) 35.00 (29.41)

Latency
There was a main effect of lead on the latency of this component, F(4,72) = 8.10, p < 0.01. Inspection of the means suggested that the maximum amplitude of this component occurred earlier at Pz than at other leads (see Figure 2 and Table 4);however, the comparisons were not significant, p > 0.05.
Discussion
As in Experiment 1, both angry and fearful expressions invoked three ERP components: (a) an early positive component, (b) a middle-latency negative component, and (c) a later positive component. In contrast to the results of Experiment 1 , the amplitude and latency of these components did not differ for the two expressions. One interpretation of these results is that the infants were unable to discriminate the angry and fearful expressions. Alternatively, the infants may have discriminated the two expressions, but because: (a) infants were equally unfamiliar with the two expressions and/or (b) the signal value of the expressions was not perceived as different (e.g., both perceived as “negative”), the responses elicited by the two expressions did not differ.
The early positive component observed in Experiment 1 was also observed in Experiment 2 . This shows that the component is invoked by angry expressions, in addition to fearful and happy ones.
General Discussion
Brain activity was recorded while 7-month-old infants saw slides of a happy and a fearful expression (Experiment 1) or an angry and a fearful expression (Experiment 2 ) . In both experiments, three ERP components were identified: (a) an early positive

FACIAL EXPRESSIONS O F EMOTION 591
component, (b) a middle-latency negative component, and (c) a late positive component. In Experiment I , the two positive components were of greater amplitude to the happy expression than to the fearful one, whereas the negative component was greater to the fearful expression than to the happy one. These results suggest that infants discriminated between the happy and fearful expressions. In contrast, in Experiment 2 none of these components differed between the angry and fearful expressions. One possibility is that infants could not discriminate between the angry and fearful expressions. Alternatively, the neural processes involved in recognizing these two expressions may not have differed because: (a) infants were equally unfamiliar with the two expressions and/or (b) the signal value of the expressions was not perceived as different (e.g., both perceived as “negative”). It is currently impossible to distinguish among these alternatives.
One question to consider is whether the differences in the ERPs invoked by the happy and fearful expressions were truly due to the differences in expression, or whether the infants were responding to some other difference between the faces. There are several reasons why it is likely that infants were responding to the differences in expressions. First, the identity of the model was the same for both expressions, thus only information related to facial expression differed between the two faces. Second, several previous studies have demonstrated that merely showing infants two differentlooking faces does not always result in a different pattern of ERPs for the two faces. For example, infants’ ERPs do not differ if they are shown two different faces with neutral expressions (Nelson & Collins, 1991).Thus, the differences in the ERPs invoked by the happy and fearful expression were probably due to the differences in expression between the two faces.
Although the results clearly indicate that infants discriminated the happy expression from the fearful one, the functional significance of the obtained ERPs must also be

Table 4 Angry-Fearful Expression: Peak Latencies in Milliseconds

a. Earlv Positive Component ( n = 19)

Lead

Expression
~~ ~
Angry mean (SD) Fearful mean (SD)

PZ
271.57 (70.26) 264.21 (72.29)

cz
284.21 (59.66) 233.68 (48.44)

FZ
285.79 (57.19) 267.37 (75.19)

T3
195.26 (35.02) 203.16 (36.52)

T4
191.58 (38.19) 200.00 (26.25)

b. Negative Component ( n = 19)

Expression

Pz

cz

Lead Fz T3

T4

Angry mean (SD) 548.21 (115.67) 531.05 (102.14) 553.15 (91.36) 456.84 (118.93) 414.74 (145.04) Fearful mean (SD) 574.21 (1 11.62) 560.00 (85.70) 569.47 (77.71) 425.79 (114.47) 451.58 (154.07)

c. Late Positive Component ( n = 19)

Expression

Lead
Pz cz Fz T3

T4

Angry mean (SD) 909.47 (159.22) 1007.89 (173.74) 1028.42 (136.11) 1083.68 (282.49) 1095.79 (277.53)
Fearful mean (SD) 899.47 (202.53) 1061.58 (162.93) 1059.47 (172.45) 1313.16 (307.25) 1085.79 (302.51)

592 NELSON AND DE HAAN
considered. In Experiment 1, a negative component thought to reflect infants’ allocation of attention was larger for the fearful face than for the happy face. This suggests that infants allocated greater attentional resources to the fearful face than to the happy one, an interpretation consistent with the results of behavioral studies showing that infants of this age look longer at fearful faces than at happy ones (e.g., Nelson & Dolgin, 1985). Thus, this negative component may be the electrophysiological manifestation of a visual preference. Whether this ERP component reflects an obligatory or a voluntary attentional response is not known. Its relatively short latency (approximately 400-700 ms) argues in favor of an obligatory process (see Nelson, 1994 for discussion). For example, it may reflect processes related to the orienting response, a phase in the engagement of attention during which the organism processes preliminary information about a stimulus (e.g., evaluates its novelty) and decides whether to allocate further cognitive resources to it (Siddle & Spinks, 1992).
The later positive component that was observed in both experiments is similar to one observed in studies of infants’ memory for facial identity. In those studies, a late positive component was observed when a stimulus was presented often enough to permit some initial encoding, but perhaps not so often as to allow for its complete recognition. This component is thought to reflect an aspect of the process of updating memory for the partially encoded stimulus (Nelson, 1994). In the present study, we predicted that (a) if 7-month-olds have relatively little experience with fearful faces in their natural environment and (b) during the experiment too few trials of fear were presented to allow full encoding and recognition, then infants should show a moreprominent late positive component for the fearful face than the happy one. Contrary to expectations, the happy face invoked a larger late positive component than the fearful one. One explanation is that infants were unable to fully encode even the happy face because the stimuli were presented very briefly (500 ms), and thus the late positive component reflects a memory-updating process. Alternatively, it is possible that this positive component may not be the same component we have observed in previous studies of infants’ memory, and thus it may reflect a different cognitive operation (i.e., it may not reflect the memory updating of a partially encoded stimulus). One argument in favor of the second interpretation is that the morphology and distribution of the late positive component observed in this study is somewhat different from the one observed in previous studies. In this study, the late positive component had a peak at midline leads and was most prominent at Cz and Pz, (see Figure 1). In contrast, in previous studies the late positive component appeared as slow-wave activity at midline leads and was most prominent at Cz and Fz (cf. Nelson & Collins, 1991). One way to test the possibility that the two components reflect different cognitive operations would be to first familiarize infants to the happy expression, and then test them with happy and fearful expressions. If the late positive component reflects memory updating that occurred because the infants did not have enough time to fully encode the happy face, then the component should disappear if the infants are first familiarized to happy. Alternatively, if the late positive component reflects some other process involved in recognizing the happy expression, then it may persist even with prior familiarization.
The early positive component observed in both experiments has not been observed in previous studies of recognition memory using ERPs (e.g., Courchesne et al., 1981; de Haan & Nelson, in press; Hofmann & Salapatek, 1981; Hofmann, Salapatek, & Kuskowski, 1981; Karrer & Ackles, 1987;Nelson & Salapatek, 1986;Nelson & Collins, 1991,1992).In many of these studies faces were used as stimuli, but the faces had neutral expressions and the task involved recognition of identity, not emotional expression (e.g.,

FACIAL EXPRESSIONS OF EMOTION 593
de Haan & Nelson, in press; Nelson & Salapatek, 1986;Nelson & Collins, 1991, 1992). Thus, one possibility is that, unlike the middle-latency negative component which has been observed by a variety of investigators using a variety of stimuli (e.g., Courchesne et al., 1981; Karrer & Ackles, 1987), the early positive component reflects a process unique to facial expressions. However, further studies that systematically compare responses to neutral faces, expressive faces, and objects other than faces are needed to determine whether this component is specific to facial expressions.
In summary, the 7-month-old infants in this study responded differentially to happy and fearful expressions, but not to angry and fearful expressions. The fact that the same ERP components were observed in both experiments (even though these components did not always distinguish between the expressions) suggests that some common neural structure(s) was activated during processing of the different expressions. It is difficult to infer the location of these neural structures based on activity recorded from only six electrode locations, however single-unit work with monkeys (Hasselmo, Rolls, & Baylis, 1989) and neuropsychological studies of brain-damaged adults (Adolphs, Tranel, Damasio, & Damasio, 1994)point to structures that lie in the temporal lobe (e.g., superior temporal polysensory area, amygdala).
Notes
This work was supported in part by a grant from the NIH (HD07151) to the Center for Research in Learning, Perception, and Cognition, University of Minnesota, and by a grant from the Natural Sciences and Engineering Research Council of Canada to the second author. Preliminary data were presented at the Society for Research in Child Development, New Orleans, LA, March 1993 and at the International Conference on Infant Studies, Paris, France, June 1994. We would like to thank the infants and parents for participating in this study, Merv Bergman for technical assistance, Kim Pearson for computer programming, and Laura Matheny and Patricia Sargent for helping to test the infants. Reprint requests should be directed to Charles A. Nelson at the Institute of Child Development, University of Minnesota, 51 East River Road,
Minneapolis, MN 55455, U.S.A. or electronic mail address CANELSON@VX.ClS.UMN.EDU.
References
Adolphs, R., Tranel, D., Damasio, H., & Damasio, A. (1994). Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature, 372, 669-672.
Barrera, M. E., & Maurer, D. (1981). The perception of facial expressions by the 3-month-old infant. Child Development, 52, 203-206.
Bentin, S., Allison, T., Puce, A., Perez, A., & McCarthy, G. (1995, March). Electrophysiological studies offace perception in humans. Poster presented at the meeting of the Cognitive Neuroscience Society, San Francisco, CA.
Bowlby, J. (1969). Attachment and loss: Vol. 1. Artachmenr. New York: Basic Books. Caron, R. F., Caron, A. J., & Myers, R. S. (1982). Abstraction of invariant face expressions in infancy.
Child Development, 53, 1008-1015. Charlesworth, W. R., & Kreutzer, M. A. (1973). Facial expression of infants and children. In P. Ekman (Ed.),
Darwin andfacial expressions:A century ofresearch in review (pp. 91-168). New York: Academic Press. Courchesne, E., Ganz, L., & Norcia, A. M. (1981). Event-related brain potentials to human faces in infants.
Child Development, 52, 804-81 I . Darwin, C. (1872). The expression o f t h e emotions in man and animals. London: John Murray. de Haan, M., & Nelson, C. A. (in press). Recognition of the mother’s face by 6-month-old infants: A
neurobehavioral study. Child Development. Ekman, P . , & Friesen, W. B. (1976). Pictures of facial affect. Palo Alto, CA: Consulting Psychologists Press. Fernald, A. (1993). Approval and disapproval: Infant responsiveness to vocal affect in familiar and unfamiliar
languages. Child Development, 64, 657-674. Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A new method of offline removal of ocular artifact.
Electroencephalography and Clinical Neurophysiology, 55, 468-484.

594 NELSON AND DE HAAN
Jasper, H . H. (1958). The 10-20 electrode system of the international federation. Electroencephalography and Clinical Neurophysiology, 10, 371-375.
Hasselmo, M. E., Rolls, E. T., & Baylis, G. C. (1989). The role of expression and identity in the faceselective responses of neurons in the temporal visual cortex of the monkey. Behavioural Brain Research, 32, 203-218.
Hofmann, M. J., & Salapatek, P. (1981). Young infants’ event-related potentials (ERPs) to familiar and unfamiliar visual and auditory events in a recognition memory task. Electroencephalography and Clinical Neurophysiology, 52, 405-417.
Hofmann, M. J., Salapatek, P., & Kuskowski, M. (1981). Evidence for visual memory in the averaged and single-trial evoked potentials in human infants. Infant Behavior and Development, 4 , 401-42 I.
Karrer, R., & Ackles, P. K. (1987). Visual event-related potentials of infants during a modified oddball procedure. In R. Johnson, Jr., J. W. Rohrbaugh, & R. Parasuraman (Eds.), Current trends in euentrelated potential research. (EEG Suppl. 40, pp. 603-608).
Kestenbaum, R., & Nelson, C. A. (1990).The recognition and categorization of upright and inverted emotional expressions by 7-month-old infants. Infant Behavior and Development, 13, 497-51 1.
LaBarbera, J. D., Izard. C. E., Vietze, P., & Parisi, S. A. (1976). Four- and 6-month-old infants’ visual responses to joy, anger, and neutral expressions. Child Development, 47, 533-538.
Lang, S . F., Nelson, C. A,, & Collins, P. F. (1990). Event-related potential responses to emotional and neutral stimuli. Journal of Clinical and Experimental Neuropsychology, 12, 946-958.
Ludemann, P. M., & Nelson, C. A. (1988). Categorical representation of facial expressions by 7-month-old infants. Developmental Psychology, 24, 492-501.
McCarthy, G., & Wood, C. C. (1985). Scalp distributions of event-related potentials: An ambiguity associated with analysis of variance models. Electroencepha1ograph.y and Clinical Neurophysiology, 62, 203-208.
Nelson, C. A. (1985). The perception and recognition of facial expressions in infancy. In T. M . Field & N. A. Fox (Eds.), Social perception in infants (pp. 101-125). Nonvood, NJ: Ablex Press.
Nelson, C. A. (1987). The recognition of facial expressions in the first 2 years of life. Mechanisms of development. Child Development, 58, 889-909.
Nelson, C. A. (1993). The recognition of facial expressions in infancy: Behavioral and electrophysiological correlates. In B. de Boysson-Bardies, S. de Schonen, P. Jusczyk, P. MacNeilage, & J. Morton (Eds.), Developmental neurocognition: Speech and face processing in the first year of life (pp. 187-193). The Netherlands: Kluwer Academic Press.
Nelson, C. A. (1994). Neural correlates of recognition memory in the first postnatal year of life. In G. Dawson & K. Fischer (Eds.), Human development and the developing brain (pp. 269-313). New York: Guilford Press.
Nelson, C. A . , & Collins, P. F. (1991). Event-related potential and looking-time analysis of infants’ responses to familiar and novel events: Implications for visual-recognition memory. Developmental Psychology, 27, 50-58.
Nelson, C. A., & Collins, P. F. (1992). Neural and behavioral correlates of recognition memory in 4- and 8-month-old infants. Brain and Cognition, 19, 105-121.
Nelson, C. A., & de Haan, M. (in press). A neurobehavioral approach to the recognition of facial expressions in infancy. In J. A. Russell (Ed.), New directions in the study of facial expression. Cambridge, MA: Cambridge University Press.
Nelson, C. A., & Dolgin, K . (1985). The generalized discrimination of facial expressions by 7-month-old infants. Child Development, 56, 58-61,
Nelson, C. A., Morse, P. A . , & Leavitt, L. A. (1979). Recognition of facial expressions by 7-month-old infants. Child Development, 50, 1239-1242.
Nelson, C. A., & Nugent, K. (1990). Recognition memory and resource allocation as revealed by children’s event-related potential responses to happy and angry faces. Developmental Psychology, 26, 171-1 79.
Nelson. C. A . , & Salapatek, P. (1986). Electrophysiological correlates of infant recognition memory. Child Development, 57, 1483-1497.
Oster, H., Daily, L., & Goldenthal, P. (1989). Processing facial affect. In A. Young & H. Ellis (Eds.), Handbook of research on face processing (pp. 107-161). Amsterdam: North Holland Press.
Schwartz, G., Izard, C., & Ansul, S. (1985). The 5-month-old’s ability to discriminate facial expressions of emotions. Infant Behavior and Development, 8, 65-77.
Serrano, J. M., Iglesias, J., & Loeches, A. (1992). Visual discrimination and recognition of facial expressions of anger, fear, and surprise in 4- to 6-month-old infants. Developmental Psychobiology, 25, 411-425.
Siddle, D. A. T., & Spinks, J. A. (1992). Orienting, habituation, and the allocation of processing resources. In B. A. Campell, H. Hayne, & R. Richardson (Eds.), Attention and information-processing in infants and adults (pp. 227-262). Hillsdale, NJ: Erlbaum.

FACIAL EXPRESSIONS OF EMOTION 595
Walker-Andrews, A . (1988). Infants’ perception of the affordances of expressive behaviors. In C. RoveeCollier & L . P. Lipsitt (Eds.), Advunces in infrncv reseurclt (Vol. 5 , pp. 173-221). Norwood, NJ: Ablex.
Walker-Andrews. A., & Lennon, E. (1991). Infants’ discrimination of vocal expression: Contributions of auditory and visual information. lnfanf Beliuuior and Deve/opmenf, 14. 13I- 142.
Wilcox, B., & Clayton, F. (1968). Infant visual fixation on motion pictures of human Paces. Joirrncrl of Experimental Child Psychology, 6 . 22-32.
Younge-Browne, G . ,Rosenfeld, H. M., & Horowitz, F. D. (1977).lnfant discrimination of facial expressions. Child Development, 48, 555-562.