Effect on Word Recognition of Time Delay between Auditory and Visual Information and Word Order of Sentences
-
- Kashimada Chiho
- Graduate School of Engnieering, Utsunomiya University
-
- Hasegawa Hiroshi
- Graduate School of Engnieering, Utsunomiya University
-
- Kamata Kazuo
- Graduate School of Engnieering, Utsunomiya University
-
- Ayama Miyoshi
- Graduate School of Engnieering, Utsunomiya University
Bibliographic Information
- Other Title
-
- 視聴覚情報間の遅延および文章の構成順序が内容の認識に与える影響
- シチョウカク ジョウホウカン ノ チエン オヨビ ブンショウ ノ コウセイ ジュンジョ ガ ナイヨウ ノ ニンシキ ニ アタエル エイキョウ
Search this article
Abstract
We investigated the effect of asynchronization between visual and auditory stimuli of speech movies upon the correct acquisition of the speech contents under various auditory-noise-conditions. We presented movies of a female announcer's face uttering short sentences to subjects who were asked to repeat the sentence aurally. All sentences consist of five words of the same word order (subject time location object verb). The utterance speed was 120 or 150 ms/mora. Audio-visual asynchronies were 0, ±1, ±2, ±4, ±8, ±16, and ±32 frames (1 frame = 1/30 s, +: visual preceding condition, -: audio preceding condition). Pink noise of -10 dB or -15 dB was imposed on the speech sound. The same procedure was done using the stimuli with no visual image (audio-only) as a control. Increase of word recognition rate compared with the level of audio-only was found within an asynchronization of ±4 frames delay indicating that the facilitation by visual information occurs within that period. On the other hand, a decrease of the recognition rate was observed when the delay is over ±8 frames, which indicates that visual information does not improve, or in some conditions, even disturbs the speech recognition. To examine the effect of word order, the same experiment was done using sentences but with a different word order. Subject performed best when presented with the order used in news reading i.e., time location subject object verb, and recognized the subject word more frequently than any others in all conditions. It is interesting that the time range of visual-facilitation is comparable with the time range of audiovisual simultaneity reported in psychophysical studies.
Journal
-
- The Journal of The Institute of Image Information and Television Engineers
-
The Journal of The Institute of Image Information and Television Engineers 62 (12), 2005-2012, 2008
The Institute of Image Information and Television Engineers
- Tweet
Details 詳細情報について
-
- CRID
- 1390282680075851904
-
- NII Article ID
- 110006991097
-
- NII Book ID
- AN10588970
-
- ISSN
- 18816908
- 13426907
-
- NDL BIB ID
- 9726294
-
- Text Lang
- ja
-
- Data Source
-
- JaLC
- NDL
- Crossref
- CiNii Articles
-
- Abstract License Flag
- Disallowed