Govor, Vol. 41 No. 1, 2024.
Original scientific paper
https://doi.org/https://doi.org/10.22210/govor.2024.41.01
Multimodality of communication – Speech and co-speech gestures
Emilija Mustapić Malenica
; University of Zadar
*
* Corresponding author.
Abstract
Under the redefined conception, multimodality represents an integrated system which contains both verbal and non-verbal modality of communication (Sandler, 2022). One of the least empirically researched non-verbal modalities are the co-speech gestures, the spontaneous body movements which convey meaning complementary with or isomorphic to the speech they accompany. This paper examines the relationship between co-speech iconic gestures and speech from a psycholinguistic perspective. Through use of experimental methods and the priming paradigm, the paper analyzes the influence of co-speech iconic gestures on speed and accuracy of processing high-frequency and high-imageability phrases in the native language and the first foreign language. The priming paradigm is based on the assumption that the exposure to the prime stimuli affects the processing of the target stimuli. All experimental trials in this research included video recordings of real actions, while the target stimuli were shown in two conditions – the audio-visual modality, in the form of a combination of speech and congruent co-speech iconic gestures, and the audio modality, in the form of speech without any gestures. Experimental trials were randomly shown in Croatian and English and interspersed with fillers. A group of native speakers of Croatian with high proficiency in English as their first foreign language was chosen as participants in the experiment. The results indicate that semantic congruence of co-speech iconic gestures does not result in the expected facilitatory effect for speed and recognition of speech processing, that is, that the participants show the fastest processing speed in the unimodal audio-only condition in both Croatian and English. The conclusions based on the collected results are explained by the low communicative contribution of speech-isomorphic co-speech gestures and from the perspective of common ground (Holler & Stevens, 2007).
Keywords
multimodality; co-speech iconic gestures; speech processing
Hrčak ID:
318869
URI
Publication date:
4.7.2024.
Visits: 0 *