Actions of attention, and attention to action: investigating the relationship between visual attention, episodic representation, and language

Abstract

Cognitive representations of episodes are likely to play an important role in the neural mechanisms representing the syntactic and semantic structure of natural language sentences. However, there is little consensus on how observed episodes are represented in the brain, and how those episode representations can be transformed to and from corresponding sentence descriptions. This work investigates the theory that there is a direct structural relationship between the representation of episodes and sentences describing those episodes, and that the underlying mechanism which informs the representation of an episode is based upon the sequence of sensorimotor actions involved in observing that episode. This gives rise to two predictions: firstly that there is a canonical sequence of sensorimotor actions involved in observing actions, and secondly, that changing that sequence leads to systematic changes in descriptions of those actions. Experimental results confirm that there is a strong default sensorimotor sequence involved in the observation of a range of transitive actions. However, in part due to the robustness of the default sensorimotor sequence, the second prediction was not confirmed.