Publication - COSMOROE: A Cross-Media Relations Framework for Modelling Multimedia Dialectics
PROFILE

COSMOROE: A Cross-Media Relations Framework for Modelling Multimedia Dialectics

Research Area:  
    
Type:  
Journal article

 

Year: 2008
Authors: Katerina Pastra
Journal: Multimedia Systems
Volume: 14
Number: 5
Pages: 299-323
Date: November
DOI: 10.1007/s00530-008-0142-0
Abstract:
Though everyday interaction is predominantly multimodal, a purpose-developed framework for describing the semantic interplay between verbal and non-verbal communication is still lacking. This lack not only indicates one’s poor understanding of multimodal human behaviour, but also weakens any attempt to model such behaviour computationally. In this article, we present COSMOROE, a corpus-based framework for describing semantic interrelations between images, language and body movements. We argue that in viewing such relations from a message-formation perspective rather than a communicative goal one, one may develop a framework with descriptive power and computational applicability. We test COSMOROE for compliance to these criteria, by using it for annotating a corpus of TV travel programmes; we present all particulars of the annotation process and conclude with a discussion on the usability and scope of such annotated corpora.
Full text:
[Bibtex]