Research

Neuronal dynamics of visual attention

What are the units of attentional selection in vision? So far, three candidates have emerged: spatial locations, simple features (e.g., color, orientation, rotation), and entire objects. For more than two decades, our research has focused on getting a better understanding of competitive interactions and neural temporal dynamics in visual selection. Topics of interest involve, among others, the relation between spatial-, object- and feature-selective attention, neuronal dynamics of feature-selective attention, and temporal neural dynamics of voluntary shifts of attention.

Spatial selection

In our seminal spatial attention studies, employing Steady State Visual Evoked Potentials (SSVEPs), we were able to uncover the time course of neural facilitation in the early visual cortex when attention was shifted to either the left or right visual hemifield (Müller et al., 1998), or that the supposedly unitary attentional spotlight may split across non-contiguous zones of the visual field (Müller et al., 2003). More recent studies revolved around the question of the distribution of pools of attentional resources between both cortical hemispheres (Walter et al., 2014; 2016) or the role of alpha oscillations as a neural signature of active stimulus inhibition in spatial attention (see below, Gundlach et al., 2020).

 

SELECTED PUBLICATIONS:

• Gundlach, C., Moratti, S., Forschack, N., Müller, M.M.: Spatial attentional selection modulates early visual stimulus processing independently of visual alpha modulations. Cerebral Cortex, 2020, 00,1–18.

• Walter, S., Quigley, C., Müller, M.M.: Competitive interactions of attentional resources in early visual cortex within or between visual hemifields during sustained visual spatial attention: Evidence for the different hemifield advantage.Journal of Cognitive Neuroscience, 2014, 26, 938-954.

• Walter, S., Keitel, C., Müller, M.M.: Sustained splits of spatial attention within versus across visual hemifields produce distinct spatial gain profiles. Journal of Cognitive Neuroscience, 2016, 28, 111-125.

• Müller, M.M., Teder-Sälejärvi, W., Hillyard, S.A.:  The time course of cortical facilitation during cued shifts of spatial attention. Nature Neuroscience, 1998, 1, 631-634.

• Müller, M.M., Malinowski, P., Gruber, T., Hillyard, S.A.: Sustained division of the attentional spotlight.  Nature, 2003, 424, 309-312.

Feature-based Selection

We were among the first, demonstrating feature-selective attention in the human brain (Müller et al., 2006). Based on that study, we developed a research program to uncover the fundamental neural dynamics in feature-based attention in the human visual cortex resulting in several publications in highly prestigious journals [LINK FULL PUBLICATION LIST]

Schematic illustration of an experimental design and results of Andersen et al. (2008). [A] Stimulus display: intermingled red and blue bars of vertical or horizontal orientation are in continuous random (Brownian) motion. [B] Bars of each type flickered at a specific frequency, thereby driving a particular SSVEP response. This allows for the concurrent measurement of the allocation of attention to each stimulus. [C] Experimental conditions: on each trial, participants attend the bars of one type to detect coherent motion targets. [D] Parallel facilitation of orientation and color: The attended stimulus shows the highest amplitude, and stimuli sharing one of the attended features also show facilitation (global effect of feature-based attention).

In more recent studies, we investigated the dynamics of attentional shifts in feature-based attention. In our first study, we showed that shifting attention to a specific feature (e.g., color) resulted in a bi-phasic process of early facilitation of the to-be-attended followed by the suppression of the to-be-unattended color (Andersen and Müller, 2010; see also Vieweg and Müller (in press) for a very recent study).  Based on that early study, we investigated whether or not suppression of the to-be-ignored feature is global as well (Forschack et al., 2017) and found that it needs (spatial) competition for a global suppression effect (Müller et al., 2018). 

 

SELECTED PUBLICATIONS:

• Vieweg, P., Müller, M.M.: Shifting attention in feature space: Fast facilitation of the to-be-attended feature is followed by slow inhibition of the to-be-ignored feature. Journal of Cognitive Neuroscience, in press.
• Forschack, N., Andersen, S.K., Müller, M.M.: Global enhancement but local suppression in feature-based attention. Journal of Cognitive Neuroscience, 2017, 29, 619-627.
• Müller, M.M., Gundlach, C., Forschack, N. Brummerloh, B.:  It takes two to tango: Suppression of task-irrelevant features requires (spatial) competition. Neuroimage, 2018, 172, 485-492.
• Müller, M.M., Andersen, S., Trujillo, N. J., Valdes Sosa, P., Malinowski, P., Hillyard, S. A.: Feature-selective attention enhances color signals in early visual areas of the human brain. PNAS, 2006, 103, 14250-14254.
• Andersen, S.K., Müller, M.M.: Behavioral performance follows time-course of neural facilitation and suppression during cued shifts of feature-selective attention. PNAS 2010, 107, 13878-13882.
• Andersen, S.K., Hillyard, S.A., Müller, M.M.: Attention facilitates multiple stimulus features in parallel in human visual cortex. Current Biology, 2008, 18, 1006-1009.

Object-based selection

In our research project concerning object-based attention, we investigated the so-called “integrated object account”, claiming that all object-constituent features will be equally well processed (O’Craven et al., 1999). As illustrated below, we found that this was not the case. Attending to one feature of an object resulted in the object’s specific facilitation and not in global facilitation of that object’s features as measured with SSVEPs.

Example taken from Brummerloh et al. (2019). [A] A rotating square that flickers in different colors evoked two distinct SSVEPs for rotation and color, respectively. Subjects attended either to rotation, color, or both features. [B] We found a significant increase for the to-be-attended (white bar) compared to the to-be-ignored feature (black bar). When subjects attended to both features, we found competitive interactions, resulting in a reduction in SSVEP amplitude compared to when they attended one of the features (grey bar).

A follow-up study looked into temporal dynamics (Brummerloh and Müller, 2019). We found a biphasic process composed of early transient integration of the constituent object features, followed by sustained feature selection mechanisms based on the amplification of the to-be-attended feature, and a time-delayed suppression of the to-be-ignored feature. Therefore, the presentation time of objects matters: Short presentation times trigger the integration process. Longer presentation times allow for feature-specific facilitation/suppression.

 

SELECTED PUBLICATIONS:

• Brummerloh, B., Gundlach, C., Müller, M.M.: Attentional facilitation of constituent features of an object does not spread automatically along object-defining cortical boundaries. Journal of Cognitive Neuroscience, 2019, 31, 278-287.
• Brummerloh, B., Müller, M.M.: Time matters: Feature specific prioritization follows feature integration in visual object processing. Neuroimage, 2019, 196, 81-93.

Attention capture by emotional images

Visual images of emotionally relevant content can be literally “eye-catching”: emotionally pleasant and unpleasant image contents rapidly draw and hold our attention for longer than neutral images, even when they are irrelevant to the activity we are concurrently conducting.

In our research, we investigate how emotional vs. neutral picture contents influence neural competition for visual processing resources in the early visual cortex, with the focus on the temporal neural dynamics of competitive interactions between task-irrelevant emotion-laden vs. neutral complex images (mostly out of the International Affective Picture System, IAPS) and a concurrent task.

To shed light on the neural time course of the attention-emotion interaction, we use various EEG markers of attention, including steady state visual evoked potentials (SSVEP), event-related potentials (ERP), and effects on induced EEG frequency bands.

 

Below are examples of two of our recent studies:

1) In one recent experiment (Bekhtereva & Müller, 2017), participants had to detect short intervals of coherent motion of randomly moving yellow squares flickering at 15 Hz to elicit the SSVEP:

[A] Stimulus examples of task-irrelevant background images. At the beginning of each trial, a scrambled background picture occurred and changed to either a neutral or unpleasant intact picture for 133 ms and then back again to its scrambled version (mask). Note: The example of an unpleasant image is taken from the GAPED database (Dan-Glauser & Scherer, 2011). [B] Time course of SSVEP amplitudes for neutral (dotted red) or unpleasant (solid red) background pictures. In the time interval between ~410 to ~830 ms, SSVEP amplitude with unpleasant images was significantly lower compared to neutral ones, indicating a withdrawal of more attentional resources from the visual task, even with briefly presented and subsequently masked visual scenes.

2) In another study, participants viewed images of unpleasant and neutral content shown in rapid serial visual presentation (RSVP) streams (Bekhtereva et al., 2018).  Here an example when images were rapidly displayed at 4 Hz (250 ms per image).

[A] Schematic illustration of RSVP streams presented at 4 Hz. At a variable time point during the trial, images could change from neutral to unpleasant content (upper panel) or vice versa (lower panel). Examples of images are taken from the GAPED database (Dan-Glauser & Scherer, 2011). [B] Time courses of 4 Hz SSVEP amplitudes for the stream of unpleasant and neutral pictures with a change in emotional content. Shortly after the onset of the emotional content change (indicated as the vertical dotted line at time point zero), SSVEP response significantly decreased for neutral relative to unpleasant scenes, and reliably enhanced when emotionally negative compared to neutral RSVP streams were shown, thus demonstrating a reliable emotion-dependent SSVEP amplitude modulation with affective image content.

In yet another study using RSVP streams with IAPS images, we were able to show that the brain was able to detect emotional regularities, i.e., the regular occurrence of an image with a particular emotional valence among an RSVP stream, i.e., every second image among other images of different emotional valence (Schettino et al., 2019). Thus, our RSVP presentation technique serves as a very powerful tool to further investigate basic neural mechanisms of attention-emotion interactions in the visual cortex (see also Bekhtereva et al., 2019) that we also used for facial stimuli (Schettino et al., 2020). The series of four experiments put a question mark behind the idea that emotional faces receive preferred processing compared to neutral faces. Our results pointed towards the direction that low-level features were responsible for SSVEP differences between neutral and emotional facial expressions.

 

SELECTED PUBLICATIONS:

• Bekhtereva, V., Craddock, M., Müller, M.M.: Affective bias without hemispheric competition: evidence for independent processing resources in each cortical hemisphere. Journal of Cognitive Neuroscience, 2020, 35, 963-976.
• Schettino, A., Porcu, E., Gundlach, C., Keitel, C., Müller, M.M.: Rapid processing of neutral and angry expressions within ongoing facial stimulus streams:  Is it all about isolated facial features? PlosOne, 2020, 15(4): e0231982. https://doi.org/10.1371/journal.pone.0231982
• Bekhtereva, V., Craddock, M., Gundlach, C., Müller, M.M.: Rapid sensory gain  with emotional distractors precedes attentional deployment from a foreground task. Neuroimage, 2019, 202, 116115, https://doi.org/10.1016/j.neuroimage.2019.116115
• Schettino, A., Gundlach, C., Müller, M.M.: Rapid extraction of emotion regularities from complex scenes in the human brain.  Collabra: Psychology, 2019, 5(1): 20. https://doi.org/10.1525/collabra.226
• Bekhtereva, V., Pritschmann, R., Keil, A., Müller, M.M.: The neural signature of extracting emotional content from rapid visual streams at multiple presentation rates: A cross‐laboratory study.  Psychophysiology, 2018, 55:e13222. https://doi.org/10.1111/psyp.13222
• Bekhtereva, V., & Müller, M.M. (2017). Bringing color to emotion: The influence of color on attentional bias to briefly presented emotional images. Cognitive, Affective & Behavioral Neuroscience. 2017, 17, 1028-1047.

Is there (pro-) active stimulus suppression/inhibition in the visual cortex?

Just recently, we started two projects questioning the role of alpha oscillations and the so-called Distractor Positivity (Pd) as neuronal signatures of (pro-) active stimulus suppression/inhibition. 

In a spatial attention experiment (Gundlach et al., 2020), we shed light on the relationship of alpha-band activity as a presumed marker of attentional selection and the attention-driven changes in the sensory gain control of the early visual cortex. For this purpose, we concurrently analyzed the time course of SSVEPs and lateralized alpha amplitudes to flickering stimuli in the left and right visual hemifield while cueing shifts of attention. In order to control for attentional deployment before the spatial shifting cue, we either instructed our subjects to attend to the central fixation cross, or to attend to both rings concurrently. We overall found robust modulations of SSVEP and alpha amplitudes due to the attentional shift. Crucially, however, our results are not in line with the assumption that alpha-modulations present a neural mechanism for implementing early sensory gain modulations. First, time courses of alpha-modulations (see figure below) were not leading attention-driven changes in SSVEP amplitudes. Second, while alpha-amplitudes increased contralateral to the unattended stimulus (in line with a presumed suppressive role of alpha modulations), SSVEP amplitudes of unattended stimuli were not suppressed as compared to pre-cue baseline but were in fact also increased. Third, and most crucially, a more fine-grained analysis revealed that, single-trial fluctuations in alpha lateralization did not affect the attentional modulation of SSVEP. We can thus conclude that the modulation of alpha-band activity does not represent a top-down mechanism that directly alters or causally controls sensory early sensory gain. Given the robust modulation of alpha-band activity by attention, the role of alpha-band activity during attention still remains a pressing question, and our findings leave space for (potential causal) alpha-attention-interactions (see Gundlach & Forschack, 2020) beyond sensory gain control in the early visual cortex.

Results taken from Gundlach et al. (2020). [A] SSVEP (bottom) and alpha (top) Gabor-filtered time courses, normalized to grand mean pre-cue baseline (-1000 to -220 ms) averaged across all conditions. Significant time-points of running t-tests are shaded in grey and were cluster corrected. Latency of the first significant time point is indicated in ms. Time zero = Shifting cue onset. [B] Single trials were sorted and grouped into five bins according to the size of alpha lateralization contralateral vs. ipsilateral to the attentional focus (i.e., the strength of the attentional alpha focus). The differences of alpha-amplitudes contralateral and ipsilateral to the attended stimulus differed across bins while the difference of the SSVEP amplitude of the attended and unattended stimulus did not differ with respect to the differences in alpha-lateralization.

In another line of experiments, we employed a typical visual search design, but all locations flickered at different frequencies. We concurrently analyzed the ERPs to the onset of the search display, SSVEP, and alpha amplitudes. Results are yet unpublished, but they clearly show that neither alpha nor the Pd is a neural signature of (pro-) active distractor suppression in the early visual cortex since we found no reduction in SSVEP amplitudes at the distractor location but an increase. This, in general, questions the idea of pro-active distractor suppression in visual search.

 

SELECTED PUBLICATIONS:

• Gundlach, C., Moratti, S., Forschack, N., Müller, M.M.: Spatial attentional selection modulates early visual stimulus processing independently of visual alpha modulations. Cerebral Cortex, 2020, 00: 1–18, doi: 10.1093/cercor/bhz335
• Gundlach, C., and Forschack, N. (2020). Commentary: Alpha Synchrony and the Neurofeedback Control of Spatial Attention. Front. Neurosci. 14, 597. doi:10.3389/fnins.2020.00597.