Abstract: It remains unclear whether there is an analogous automatic deviant-related negativity elicited outside the auditory modality even though the MMN can be elicited in auditory modality. The present study employed simultaneous audio-visual stimulus in the oddball paradigm to re-examine the effects of attention on visual mismatch negativity in audiovisual perception. The electrical brain activities were recorded from normal, from normal participants subjects. Stimuli consisted of a set of four audio-visual stimuli that are distinguished by frequencies (Hz) for audio and features for visual appearing on the computer screen. ANOVA showed statistically significant of the interaction between electrode site and modality. The difference waves with 100-200 msec latency at the anterior sites were markedly different to the posterior sites. The emergence of posterior negativity in the audio-visual modality might not be attributed to visual discrimination process as it did not appear in the visual modality.
INTRODUCTION
Mismatch negativity (MMN) component of event-related potentials (ERPs) is theoretically elicited in the auditory cortex when incoming sounds are detected as deviating from a neural representation of acoustic regularities. It is mainly generated in the auditory cortex (Scherg et al., 1989) occurring between 100 to 250 msec and thus long been regarded as specific to the auditory modality (Nyman et al., 1990; Naatanen, 1992). The automatic change-detection system in the human brain as reflected by the MMN requires the storage of the previous state of the acoustic environment for detecting an incoming deviating sound (Naatanen, 1992; Brattico et al., 2002). MMN implies the existence of an auditory sensory memory that stores a neural representation of a standard against which any incoming auditory input is compared (Ritter et al., 1995). In the auditory modality, MMN is an automatic process which occurs even when the subject’s attention is focused away from the evoking stimuli (Naatanen, 1992). Its onset normally begins before the N2b-P3 complex which occurs when attention is directed to the stimuli. The duration of MMN varies with the nature of the stimulus deviance but it invariably overlaps N2b when the latter is present (Tales et al., 1999).
Although, it is clear that the MMN can be elicited in auditory modality in the absence of attention, it remains somewhat unclear whether there is an analogous automatic Deviant-related Negativity (DRN) elicited outside the auditory modality. Previous study (Naatanen, 1990) has stated that the automatic detection of stimulus change plays a part in directing attention to events of biological importance. If this is the case, one would expect a similar mechanism to operate in the visual modality. Several studies have shown that visual stimuli deviating from repetitive visual standards can also elicit a visual analogue of the MMN in the same latency range. This visual MMN seems to be mainly generated in occipital areas (Czigler et al., 2004; Yucel et al., 2005) with possibly a more anterior positive component (Czigler et al., 2002; Heslenfeld, 2003). In addition, cross-modal attention studies clearly showed that deviant visual stimuli elicited MMN, largest over the inferior temporal cortex. This visual MMN increased in amplitude with attention, but it was also evident during inattention (Alho et al., 1992; Woods et al., 1992). Recently, Cammann (1999) showed a widely distributed MMN change between 150 and 350 msec, with a parietal maximum suggesting that this MMN may occur in the visual modality. In the present study, simultaneous audio-visual stimulus in the oddball paradigm was used to re-examine the effects of attention on visual MMN in audiovisual dimensions.
MATERIALS AND METHODS
Participants: Twelve right-handed normal subjects (6 males and 6 females) with a mean age of 24.83 (SD = 3.54) were participated in the experiment. All participants had normal hearing and corrected to normal vision (self reported). None of them had more than three years of formal musical training and none had any musical training within the past five years. All participants had no history of neurological or psychiatric history. After a complete description of the intended study, written informed consent was obtained.
Stimuli: Stimuli consisted of a set of four audio-visual stimuli that are distinguished by frequencies (Hz) for audio and features for visual appearing on the screen. The duration of the stimuli were 300 msec. The stimulus system (STIM, Neurosoft, Inc. Sterling, USA) was employed for controlling the presentation of the stimuli. An oddball paradigm was chosen for presenting randomized stimulus sequences consisting of all four sets of equiprobable audio-visual stimuli: the deviant was ‘X’ with 1800 Hz tone (Visual Target Audio Target; hereafter, VTAT) in 10% probability and the standard was ‘Y’ with 800 Hz tone (Visual Non-target Audio Non-target; hereafter, VNAN) in 70% probability were presented as preferred-deviant to be able to check that participants were attending the stimuli. Additionally, the deviant ‘X’ with 800 Hz tone (Visual Target Audio Non-target; hereafter, VTAN) and the deviant ‘Y’ with 1800 Hz tone (Visual Non-target Audio Target; hereafter, VNAT) were used in 10% probabilities (Fig. 1). While visual stimuli were presented on the computer screen, acoustic/audio stimuli were delivered binaurally to the subjects through plastic tubes and earpieces. Sound density was adjusted to be 85 dB above the participants’s hearing threshold.
Procedures: The experiment was consisted of 3 blocks and each block had 300 trials. Every stimulus was presented with 300 msec exposure duration and inter- stimulus interval (ISI) was 1800 msec (from audio/visual stimuli onset to onset) in every condition.
| Fig. 1: | Schematic presentation of the stimuli in bisensory conditions. Stimuli in different modalities are presented simultaneously |
Participants sat in an electrically shield and soundproofed room with the response buttons under their hands. The participants had to press the button on the response pad when the deviant (VTAT) was presented and ignore any other types of stimuli. Prior to the experimental session, a practice block was administrated to ensure that the participants understood the task.
Electroencephalogram recording: EEG data was recorded from a Quick-Cap equipped with 128 channels according to the international 10-20 system using Scan system (Scan 4.2, Neurosoft, Inc. Sterling, USA). Linked mastoids were used as reference. Eye movements were monitored with two EOG electrodes. Impedance was maintained at 5 kΩ or less. During the experiment, EEG was amplified with a bandpass of 0.05-100 Hz, sampled at 1000 Hz and stored for off-line analysis. ERPs were averaged separately for each types of stimulus. They were digitally filtered with a bandpass of 0.1-30 Hz. The averaging epoch was 900 msec and the 100 msec before the onset of the presenting stimuli served as baseline. The artifact-free epochs were filtered at 0.1-15 Hz, baseline corrected and averaged. The artifacts rejection was conducted in all channels with threshold of ±100 μV before averaging.
Electroencephalogram data analysis: After the data recordings, the EEG was segmented into 1000 msec epochs, including the 100 msec pre-stimulus period. The EEG epochs contained amplitudes exceeding ±100 μV at any EEG channels were automatically excluded from the averaging. The average waveforms obtained from the standard and deviant stimuli were digitally filtered by a 0.1-15 Hz band-pass filter and finally baseline-corrected. To analyze the deviant-related components, difference potentials were calculated where responses elicited by the VNAN stimuli were subtracted from responses to VTAN and VNAT stimuli after stimulus onset referred to visual (Vi) modality as in Eq. 1 and auditory (Au) modality as in Eq. 2, respectively:
(VNAN)-(VTAN)
= (Vi) |
(1) |
and
(VNAN)-(VNAT)
= (Au) |
(2) |
In the audio-visual (AV) modality, VTAT minus VNAN difference was also calculated as in Eq. 3:
(VNAN)-(VTAT)
= (AV) |
(3) |
MMNs were statistically assessed by two-tailed t-tests comparing the averaged amplitude of the deviant minus standard difference waveform to zero in the 40 msec time-window around the latency of the peak in the grand-average responses. To compare these components, MMN amplitudes were further assessed via two-way Analyses of Variance (ANOVA) with repeated measurements. The factors were modality (three levels: Vi, Au and AV) and electrode site (two levels: anterior sites at F3, Fz, F4, C3, Cz, C4 and posterior sites at P3, Pz, P4, O1, Oz, O2).
RESULTS
Reaction times and response accuracy (mean and standard deviation: SD) was shown that the mean reaction time for VTAT at 0.043 msec where as the percentage of hit rate correction was high (Table 1). Figure 2 represents the grand-average deviant-related components in the Au, Vi and AV modalities producing deviant-related negativities (DRNs). DRNs were divided into an early DRN1 around 100-200 msec and a late DRN2 around 200-300 msec.
Two-way repeated measures ANOVA shows that the interaction between electrode site and modality of MMN amplitudes at 100-200 msec of all modalities was statistically significant [F(11,429) = 8.27, p<0.0001]. At 200-300 msec, significant levels were also reached in the same interaction for N2b component [F(11,429) = 6.50, p<0.0001]. As shown in Figure 2, the difference waves with 100-200 msec latency at the anterior sites were markedly different to the posterior sites. Additionally, there was no MMN elicitation for the Vi modality at the posterior sites compared to the Au and AV modalities. Therefore, the MMN mean amplitudes of all modalities were compared. Two-way repeated measures ANOVA shows that the interaction between posterior electrode site and modality was statistically significant [F(17,663) = 27.52, p<0.0001] and significant level was also reached in the interactions between anterior electrode site and modality [F(17,663) = 52.37, p<0.0001].
| Table 1: | Mean reaction time and response hit rates for the VTAT condition |
| Fig. 2: | Grand-average of the deviant-related components obtained by subtracting the event-related potential of standard stimuli from that of deviant stimuli in auditory (Au), visual (Vi) and audiovisual (AV) modalities |
DISCUSSION
The main finding of the study indicates that the prominent response to the Au, Vi and AV modalities produces deviant-related negativities. DRNs were divided into early DRN1 (or MMN) and late DRN2 (or N2b).
According to the previous studies showing that MMN appears between 100 to 250 msec (Naatanen, 1992) and the characteristics of DRN2 match with those of N2b component (Berti and Schroger, 2001), the present study thus associated DRN1 mainly with MMN in which we focus in this report and DRN2 with a mixed wave of MMN and N2b. As shown in Fig. 2, the difference waves with 100-200 msec latency at the anterior sites were markedly different to the posterior sites. There was no MMN elicitation for the visual modality at the posterior electrode sites compared to the auditory and audiovisual modalities. This result was consistent with the previous study showing no posterior negativity elicitation in the difficult discrimination task (Czigler and Balazs, 2001). Moreover, the present result extends previous findings (Berti and Schroger, 2001; Alho et al., 1994) showing that the deviance related ERP effects in vision could be separated from automatic processing of other stimulus features. The emergence of posterior negativity in the present study is not to be attributed to visual discrimination process.
The present result supports the view proposed by Czigler et al. (2004) that “no MMN appears to occur in the visual modality”. In addition, Alho and colleagues proposed that if a visual MMN exists, its elicitation may have a higher threshold than auditory MMN which evoked by any discriminable change (Alho et al., 1994). Moreover, the identical N2b components were elicited by Vi and AV modalities, whereas in case of Au modality, latency of this component was longer than that of the Vi and AV modalities. The N2b effect suggests the attention-related rechecking of the outcome of within-modality analyses. Such activity would have been different upon the different discrimination demand (Smid et al., 1999; Czigler and Balazs, 2001). The process underlying N2b component thus performs independent within-dimension selection (Smid et al., 1999). The shorter N2b latency to AV further suggests that this component is a correlate of processes following the elementary discrimination processes, instead of being an on-line correlate of such processes (Czigler and Balazs, 2001). Consequently, the findings in the present study are consistent with the previous study showing that the deviant-related negativities consist of two successive components, the earlier being generated at the auditory cortex and the latter at the frontal areas (Rinne et al., 2000).
CONCLUSION
The present study demonstrates the audiovisual interaction following elementary within-modality discrimination processes. MMN and N2b effects suggest the attention-related rechecking of the outcome of within-modality analyses. The task-related processing of audio and visual features was independent and one modality might influence the processing of the other. This findings support the view that the processing of a feature, hierarchically dependent on another feature in the condition of audio-visual perception.