Time to read, time to watch: Eye movements and information processing in subtitled films

Watching foreign-language media is part of our everyday lives. Techniques such as subtitling or dubbing provide the verbal information to foreign linguistic communities. To study the implications of these techniques on the information that viewers gather from the audiovisual scene, 24 English speakers watched 8 movie clips in 4 possible versions of language and subtitles: (EA-NS) English audio without subtitles; (SANS) Spanish audio, without subtitles; (SA-ES) Spanish audio with English subtitles; and (EA-SS) English audio with Spanish subtitles. Eye movements were recorded and analyzed relative to dynamic regions of interest. A questionnaire assessed the recall of visual and verbal information. Results show that the region corresponding to the subtitles was fixated significantly longer when clips were presented in SA-ES version compared to other conditions, confirming that participants read the subtitles. Nevertheless, participants spend most of the time fixating actor faces, and to a lesser extent other items appearing in the scene. The effect of presenting subtitles has an effect on the time dedicated to observe faces, however the decrease in the observation time does not affect the recall performance for scene items and dialogue information, suggesting that dubbing and subtitling are equivalent in terms of the important narrative information processed.

Book of Abstracts of the 17th European Conference on Eye Movements, 11-16 August 2013, in Lund, Sweden