Publication - Multimodality in TV interviews: development and exploitation of multimedia corpora in the representation and modeling of multimodal communication [in Greek]
RESEARCH

Multimodality in TV interviews: development and exploitation of multimedia corpora in the representation and modeling of multimodal communication [in Greek]

Research Area:  
    
Type:  
Phd Thesis

 

Year: 2012
Authors: Maria Koutsombogera
University: National and Kapodistrian University of Athens
DOI: 10.12681/eadd/28844
Abstract:
The research presented in this thesis explores the multimodal aspects of communication in the genre of television interviews. Specifically, it unfolds a cross-disciplinary, corpus-based approach on the communicative role of the non-verbal expressions (gestures, facial expressions, body posture) employed by the interview participants and their interplay with speech in terms of their contribution to the structure, content and perception of a participant’s message. Taking into account previous and state-of the-art research on the analysis of non-verbal interaction, the thesis addresses both methodological and research issues. The former deal with the development of a multimodal corpus and the related design of an appropriate coding scheme applied for the annotation of the data via a computational tool catering for the handling of the audiovisual input and the respective structured output format. The resulting annotated corpus attestations are studied in detail in an attempt to find evidence about their systematic role as well as to explore non-verbal communicative patterns through which the speakers regulate their interaction. The analysis focuses on the distribution and frequency of the various features, the classification of non-verbal expressions according to semiotic, semantic and pragmatic criteria as well as their mapping to semantic equivalent and syntactic constituents, shedding light to the detailed ways in which the linguistic and the non-verbal system interact in the effective, synchronised communication of information. Furthermore, the thesis investigates the effect of the situational and conversational setting of each interview, as well as possible cross-cultural diversities, on the interactional behavior of the participants and their preferences. Finally, it concludes with recommendations on the exploitation of the resulting research and future work.
[Bibtex]