Research Sketch: Järvikivi, Schimke, & Pyykkönen-Klauck (2019)

HIGHLIGHTS

• Pronominal reference to entities that are left implicit proceeds without difficulty when the entities are conceptually central to the event and inferable within the current mental model.
• Understanding reference to conceptually more distant entities is significantly more time consuming, whether these entities are explicitly mentioned or implicit (as shown by visual world eye movement data).
• Importantly, the gaze data further indicates that people are unable to immediately use the visual context to resolve implicit reference.
• The visual environment is only integrated when it is relevant or acts as a fall-back when the referential search within the current discourse representation fails.

ABSTRACT

We often use pronouns like it or they without explicitly mentioned antecedents. We asked whether the human processing system that resolves such indirect pronouns uses the immediate visual-sensory context in multimodal discourse. Our results showed that people had no difficulty understanding conceptually central referents, whether explicitly mentioned or not, whereas referents that were conceptually peripheral were much harder to understand when left implicit than when they had been mentioned before. Importantly, we showed that people could not recover this information from the visual environment. The results suggest that the semantic–conceptual relatedness of the potential referent with respect to the defining events and actors in the current discourse representation is a determining factor of how easy it is to establish the referential link. The visual environment is only integrated to the extent that it is relevant or acts as a fall-back when the referential search within the current discourse representation fails.