PROFILE
Overview of the CLEF 2008 Multilingual Question Answering Track
Year: | 2009 | ||||
---|---|---|---|---|---|
Authors: | Pamela Forner; Anselmo Penas; Inaki Alegria; Corina Forascu; Nicolas Moreau; Petya Osenova; Prokopis Prokopidis; Paulo Rocha; Bodgan Sacaleanu; Richard Sutcliffe; Erik Tjong Kim Sang | ||||
Publisher: | Springer-Verlag | ||||
Editor: | Carol Peters and Thomas Deselaers and Nicola Ferro and Julio Gonzalo and Gareth J. F. Jones and Mikko Kurimo and Thomas Mandl and Anselmo Pe?as and Vivien Petras | ||||
Address: | Berlin, Heidelberg | ||||
Pages: | 262-295 | ||||
ISBN: | 3-642-04446-8, 978-3-642-04446-5 | ||||
Abstract: | The QA campaign at CLEF 2008, was mainly the same as that proposed
last year. The results and the analyses reported by last year’s participants
suggested that the changes introduced in the previous campaign had
led to a drop in systems’ performance. So for this year’s competition
it has been decided to practically replicate last year’s exercise.
Following last year’s experience some QA pairs were grouped in clusters.
Every cluster was characterized by a topic (not given to participants).
The questions from a cluster contained co-references between one
of them and the others. Moreover, as last year, the systems were
given the possibility to search for answers in Wikipedia as document
corpus beside the usual newswire collection. In addition to the main
task, three additional exercises were offered, namely the Answer
Validation Exercise (AVE), the Question Answering on Speech Transcriptions
(QAST), which continued last year’s successful pilots, together with
the new Word Sense Disambiguation for Question Answering (QA-WSD).
As general remark, it must be said that the main task still proved
to be very challenging for participating systems. As a kind of shallow
comparison with last year’s results the best overall accuracy dropped
significantly from 42% to 19% in the multi-lingual subtasks, but
increased a little in the monolingual sub-tasks, going from 54% to
63%. |
||||
[Bibtex] |