Object-Based Benefits without Object-Based Representations

Influential theories of visual working memory have proposed that the basic units of memory are integrated object representations. Key support for this proposal is provided by the ‘same object benefit’: it is easier to remember multiple features of a single object than the same set of features distributed across multiple objects. Here we replicate the object benefit, but demonstrate that features are not stored as single, integrated representations. Specifically, participants could remember ten features better when arranged in five objects compared to ten objects, yet memory for one object feature was largely independent of memory for the other object feature. These results rule out the possibility that integrated representations drive the object benefit, and require a revision of the concept of object-based memory representations. We propose that working memory is object-based in regard to the factors that enhance performance, but feature-based in regard to the level of representational failure.

It is much easier to remember a set of visual features that are arranged into a small number of objects than to remember the same set of features distributed across multiple objects (Delvenne & Bruyer, 2004;Luck & Vogel, 1997;Olson & Jiang, 2002;Vogel et al., 2001;Wheeler & Treisman, 2002).For example, it is easier to remember 5 colors and 5 orientations that appear in the same 5 objects than it is to remember the same 10 features on separate objects (Olson & Jiang, 2002;Xu, 2002) (Figure 1).The finding that working memory improves with fewer discrete objects (Olson & Jiang, 2002;Xu, 2002) has been used as evidence that the representations that underlie working memory are object-based (Luck & Vogel, 1997;Vogel et al., 2001).According to this theory, working memory can store a small, fixed number of objects, and therefore integrating multiple features into a single object representation enables more features to be stored.
However, the improvement for object displays does not necessarily imply the storage of integrated, object-based representations.It is possible that there is a cost to representing additional objects-hence the benefit of encoding information from fewer objects-but that memory representations themselves consist of non-integrated collections of features (Bays, Wu, Husain, 2011;Kyllingsbaek & Bundesen, 2007;Wheeler & Treisman, 2002;Fougnie & Alvarez, 2011;Stefurak and Boynton 1986; but see Gajewski & Brockmole, 2006;Irwin & Andrews, 1996).To directly address this question, it is necessary to measure memory for multiple features of the same object.If the object-benefit were due to the features being stored as an integrated object, then memory of one feature would be dependent on, and indicative of, whether that item's other feature was stored.Thus, in the present study we measured memory for color and orientation when both features appeared on the same object, versus when the features appeared on different objects, while probing memory for both color and orientation within the same trial.This procedure enables us to assess whether there is an object-benefit, and whether features are stored as an integrated unit, within the context of a single study.

Methods
Twenty-one participants were asked to remember five colors and five orientations where each feature was in a distinct object (10-object condition) or where objects were defined by color-orientation conjunctions (5-object condition).Conditions were presented in separate 90-minute sessions (540 trials) and session order was random.

5-object displays-
Five triangles defined by color and orientation were presented in evenly spaced position along an imaginary ring (3.5° radius) from fixation.
A trial consisted of a 1200ms sample presentation, followed by a 900ms retention interval, followed by non-speeded color and orientation reports (in a random order) (Figure 1).During feature reports a solid white circle indicated the to-be-reported location.
Participants were asked to adjust the task-relevant feature to match the sample item corresponding to the cued location.Participants adjusted probe color by selecting a value along a circular color wheel (6° radius, centered on fixation).The selected value was determined by the angle of the cursor position in reference to fixation.While cursor position was hidden, participants knew the currently selected value since the color of the probe stimulus was continuously updated to the selected color.For orientation reports, the orientation of the small angle was determined by mouse position in reference to the probed item.The probe stimulus was continuously updated to match the selected orientation.A black indicator line appearing on the outer edge of the response wheel indicated the selected color or orientation value.To encourage participants to store features in an integrated fashion, in the 5-object condition participants adjusted the color and orientation of a single item before submitting a response (see also Fougnie & Alvarez, 2011;Bays, et al., 2011).In this condition, participants could switch between adjusting color or orientation by clicking the mouse.Feedback in degrees of error for each feature was provided after both reports.
For data analysis we utilized the distribution of errors to estimate the proportion of guess and memory responses and the precision of memory responses for each feature (Zhang & Luck, 2008).Figure 2 shows an error distribution for a representative participant's color response errors.Our analysis method assumes that a participant responds in one of two states: memory or guess response.On trials in which a participant guesses the response will be random relative to the true value.Over many trials this will lead to a uniform distribution of response error.On trials in which a participant responds from memory we assume that responses will be normally distributed around the correct value, with the width of this distribution signifying the fidelity of memory.We use the observed error distribution to find the best fitting weighted mixture of a uniform and a circular normal distribution (for an example see the red line in Figure 2) (using maximum likelihood estimation).The estimated weighting of the uniform versus normal distribution corresponds to the proportion of guess and memory responses in the data.The estimated width of the normal distribution corresponds to the precision of memory for that condition.

Results & Discussion
Each condition was modeled as a weighted mixture of a circular normal and a uniform distribution in order to estimate the proportion and precision of memory responses (Figure 2; Zhang & Luck, 2008).Figure 3A shows the model-fitted response error distributions for each condition derived from averaging the best fitting parameter values for each participant.There were higher proportions of memory responses (and fewer guess responses) for 5-object displays (color 62.0%, orientation 47.2%) than 10-object displays (33.0%color, 23.0% orientation, both t's > 6, both p's < .001)(Figure 4A shows the values averaged across features) 1 .The proportion of memory responses for 5-object displays was approximately double that of 10-object displays (103.1% increase) even though feature load was equivalent.Displays with 5 objects also had slightly improved fidelity (lower standard deviation of the memory response distribution) for orientation (21.4° versus 26.1°, p < .005),but not for color (p = .41).These findings replicate past work showing that participants can store twice as many features when two features are conjoined into a single object (Luck & Vogel, 1997;Olson & Jiang, 2002;Vogel, et al., 2001;Xu, 2002).
We then asked whether this benefit for fewer objects occurred because participants are able to store twice as many features when an object shared two features because those features were integrated into a single representation.If so, then when a participant guessed for one feature of an object, they would be highly likely to guess on the object's other feature as well.To test this possibility, we performed an additional modeling analysis that only included trials where participants guessed on the objects other feature (where guesses were classified as responses more than three standard deviations away from the correct value; Fougnie & Alvarez, 2011).Figure 3B shows the participant averaged response error distributions both for analyses including all trials (solid lines) and for analyses including only the trials in which participants guessed on the object's other feature (dashed lines).
Integration would predict that the dashed lines would be uniform and have no central Gaussian component (0% memory responses).However, we found that participants were only slightly more likely to guess when they did not know the objects' other feature.We observed a high proportion of memory responses for color given an orientation guess (52%, only a 15% drop) and orientation given a color guess (33%, only a 32% drop), providing strong evidence for largely independent storage of features.These findings suggest that the improvement in performance for fewer objects did not arise entirely from integrating features into a single object representation.If feature integration fully explained the improvement in performance in the 5-object condition then the percent increase in memory responses in the 5-object condition should be equivalent to the degree to which representations were integrated.Yet, the estimate of degree of integration of features (22.8%, estimated by measuring the average decrease in memory responses including only trials were participants guess on the other feature) was drastically lower than the percent increase in feature storage capacity (103.1%),t(20) = 8.1, p< .001(Figure 4B).
Furthermore, these values -degree of integration and percent increase in storage capacity -were not even correlated with each other within subjects (r 2 = .11,p> .1) 2 (Figure 5).
We cannot conclude that the two measures are completely unrelated, particularly since small correlations may be difficult to observe reliably with only 21 participants.However, the lack of any correlation is further evidence that the object benefit is (at most) minimally influenced by the degree of feature integration.
We should note that while some experiments have observed weaker object benefits than found in the present study (e.g.Olson & Jiang, 2002) these effects were still greater than the integration observed here.Thus, our findings suggest that even a 'weak-object' benefit (Olson & Jiang, 2002) is not necessarily consistent with partial integration of features into object-based representations.
One concern is that low integration is due to memory failures that arise during the response period, with the effort of trying to retrieve the first feature causing forgetting of the second feature.An increase in the guess rate for the second response could give rise to apparent evidence of independent feature memory because retrieval-induced guesses for the second response could occur on trials in which participants did not guess for the first response.Importantly, the change in guess rate between the first (57%) and second (54%) response, while significant (p=.005), was too small to explain the feature independence that was observed.Furthermore, even when participants guessed on their first response, they still showed good memory for the second response.Indeed, participants were only 24.8% more likely to guess on the second response if they guessed on the first response (compared to how often participants guessed on the second response in all trials) 3 .Since it can be safely assumed that the act of retrieving the second feature would not impair the subject's ability to answer about the first, the independence of feature memory can be attributed to nature of the memory representations rather than to an artifact of the testing procedure.Another possible explanation for the independence between features is that the probe in the previous trial might lead participants to store fewer colors or orientations than they are able to store 4 .For example, participants may prioritize orientation at the expense of color in trials that immediately follow trials in which the first response probe asked for an orientation judgment, and vice-versa.However, we found equivalent independence for each feature regardless of the report order of the previous trial (color, p = .19;orientation, p = .59).
The memory load for the 5-object condition was slightly greater than the capacity of four items estimated by many studies on the limits of working memory (e.g.Cowan, 2001;Luck & Vogel, 2001, Vogel et al., 2001).We considered whether the independence between features was due to the supra-capacity demands of the task.Specifically, one might suggest that while a participant can store four integrated object representations, any additional information would be retained in a feature-independent fashion.However, this account would predict much higher estimates of integration than were observed (at least 80%).In fact, given the overall performance (54.6% memory responses) and the degree of integration (22.8%) the maximum number of integrated representations consistent with the present data is less than one item (.62; 54.6% * 22.8% * 5).To further address this concern we conducted an additional experiment on 11 new participants that compared performance for remembering 8 features in 4 or 8 objects.Importantly, while the load of the 4-object condition was now within standard measures of working memory capacity, we still observed an object benefit (95.1%) that was larger than the degree of integration (23.1%), t(10) = 3.87, p < 005.The degree of integration was equivalent across studies (t = .01,p = .99)suggesting that integration is not influenced by the memory load.
Previous studies have shown this independence across features to be resilient to methodological details such as encoding duration (Fougnie & Alvarez, 2011) and the method of probe response (Bays et al., 2011;Fougnie & Alvarez, 2011).Yet, by showing a large object-based benefit and largely independent feature storage in the same context the present findings go significantly beyond previous work in placing constraints on the cause of this feature independence and to rule out alternative explanations.Consider that any aspect of the 5-object condition that would lower measures of integrated features (such as response order effects) would also produce an equivalent drop in the observed object benefit, were that benefit driven solely by the storage of integrated objects.Therefore, by showing a large object-based benefit and largely independent feature storage in the same context the present findings go significantly beyond previous work and cannot be reconciled with the standard view that the object benefit reflects multiple features being integrated into a single object representation.
To explain how we can observe evidence for an object benefit in the same context as evidence for independent failures of memory we propose a major departure from previous theories of visual working memory, which have proposed that memory limitations arise entirely from the availability of some limited-commodity resource that's either quantized into slots (Zhang & Luck, 2008) or continuously divisible (Alvarez & Cavanagh, 2004;Bays & Husain, 2008;Wilkin & Ma, 2004).Here we propose that stochastic noise processes impose an important additional constraint on memory -above and beyond any limits due to the availability of a limited commodity (slots or resources).On this account, the survival of memory representations is probabilistic, and therefore can be different even for two objects that received equal resources.The co-occurrence of an object-benefit without the integration of object features can be accommodated by this probabilistic feature store framework with two assumptions: (1) the number of objects represented is one of many factors that increase the amount of stochastic noise in the system, and (2) feature representations can fail independently (i.e., are not integrated).
The present results suggest that there is reduced likelihood of representational failure in the 5-object condition relative to the 10-object condition but that the locus of failure is still independent features rather than coherent objects.One reason why representations may be more likely to fail when attempting to store more objects is that our working memory system may be assisted by a top-down reactivation or rehearsal mechanism that acts in an object-based fashion to decrease representational failure (Schneider, 1999).
While more items competing for representation may lead to an increased probability of representational failure for all features, we suggest that these failures are stochastic and occur independently (Huang, 2010).This probabilistic feature store account fits well with neural models of memory representation.For example, there is strong evidence that the biophysical processes that underlie maintenance of memory representations are stochastic and noisy (Ma, Beck, Latham, & Pouget, 2006;Rolls, 2008;Rolls & Deco, 2010;Tegner, Compte, & Wang, 2002;Treves, Panzeri, Rolls, Booth, & Wakeman, 1999;Wang, 2001) and that the neural substrates of memory for precise perceptual judgments are the neural regions involved in coding stimulus identity during perception (Harrison & Tong, 2009;Serences, et al., 2009).If representations for different features of objects are sustained in independent, noisy neural populations but are assisted by a reactivation mechanism that acts in an object-based fashion, then this could produce object-based benefits without storage of object-based representations.On this view, features that are coded independently perceptually and neurally (so called separable dimensions, Cant et al., 2008;Drucker, Kerr, & Aguirre, 2009;Garner, 1974;Livingstone & Hubel, 1988), such as color and orientation may have largely independent instances of representational failure, whereas integral features such as height and width (Garner, 1974) may fail together.This account places the locus of working memory limitations at storage rather than at encoding or perception.Past work has suggested that object-based limitations arise during storage, not encoding, in part because the effects is observable across a range of encoding intervals (Vogel, et al., 2001) and across methods of stimulus presentation (e.g.placing features at the same spatial positions; Fougnie, Asplund, & Marois, 2010;Lee & Chun, 2001).Indeed, in a separate study on 6 participants we still observed a sizeable object benefit when we doubled the encoding duration for each condition (2400ms; p < .05).However, it's possible that there are encoding limitations that are not resolved by the amount of time for encoding information, and that these limitations are reduced with fewer discrete objects.The present results would still imply that the factors that influence encoding capability would be distinct from the nature of the encoding representations.Specifically, it is possible that the encoding of features may be probabilistic and independent (Vul & Rich, 2010;Kyllingsbaek & Bundesen, 2007) but that encoding is more likely to be successful for each feature when there are fewer objects.Thus, regardless of the source of the object benefit, by demonstrating an object benefit in the same context as independent failures of features, the present study highlights that determining which factors influence representational failure does not necessarily inform us about the nature of the underlying representations.This insight will require modification to existing models and theories (e.g.Cowan, 2001;Luck & Vogel, 1997) and more broadly in how we conceive of representational limitations.

Footnotes
1 A large object benefit was also observed when comparing responses for the feature that was probed first ruling out retrieval costs (Woodman & Vecera, 2011) as driving the object benefit.
2 Both measures showed split-half reliability-measures for odd trials were correlated with measures for even trials, (degree of integration, r = .59,p < .05;percent increase in capacity, r = .43,p < .05).Note that the split-half reliability measure drastically reduces the number of trials contributing to the model and will underestimate the maximum correlation you could observe between the measures.To increase the number of trials included in the estimate of degree of integration the criteria for classifying guess trials was reduced to 1 standard deviation. 3The measure of integration in this analysis is determined by how often participants guessed on the second response.Therefore, worse performance for the second response could lead to an overestimate of the degree of integration.Importantly, this analysis will never underestimate the degree of integration.

Figure captions
Figure 1: Trial timeline (left to right) for the 5-object condition (top row) and 10-object condition (bottom row).On some trials participants were asked to report color before orientation.

Figure 2 :
Figure 2: Histogram of color response errors for the 10-object condition of a representative

Figure 3 :
Figure 3: Modeled response error distributions using the average of the best fitting parameter

Figure 4 :
Figure 4: (A) The parameter estimates of the percent memory responses for the 10-object and 5-

Figure 5 :
Figure 5: Scatter plot of participants increase in memory responses for 5-object compared to 10-