Overview of the CLEF 2006 Multilingual Question Answering Track

Magnini, Bernardo; Giampiccolo, Danilo; Forner, Pamela; Ayache, Christelle; Jijkoun, Valentin; Osenova, Petya; Peñas, Anselmo; Rocha, Paulo; Sacaleanu, Bogdan; Sutcliffe, Richard F. E.

doi:10.1007/978-3-540-74999-8_31

Cited by 44 publications

(33 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Last year we promoted an architecture based on Textual Entailment trying to bring research groups working on machine learning to Question Answering. Thus, we provided the hypothesis already built from the questions and answers [6] (see Figure 2). Then, the exercise was similar to the RTE Challenges [1] [2] [3], where systems must decide if there is entailment or not between the supporting text and the hypothesis.…”

Section: Question Answering Trackmentioning

confidence: 99%

“…Development collections were obtained from the QA@CLEF 2006 [6] main track questions and answers. Table 1 shows the number of questions and answers for each language together with the percentage that these answers represent over the number of answers initially available, and the number of answers with VALIDATED and REJECTED values.…”

Section: Development Collectionsmentioning

confidence: 99%

“…Thus, this baseline can be seen as the average proportion of correct answers per question group (6). …”

Section: Evaluation Of the Answer Validation Exercisementioning

confidence: 99%

See 2 more Smart Citations

Overview of the Answer Validation Exercise 2008

Rodrigo

Peñas

Verdejo

2009

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

The Answer Validation Exercise at the Cross Language Evaluation Forum is aimed at developing systems able to decide whether the answer of a Question Answering system is correct or not. We present here the exercise description, the changes in the evaluation methodology with respect to the first edition, and the results of this second edition (AVE 2007). The changes in the evaluation methodology had two objectives: the first one was to quantify the gain in performance when more sophisticated validation modules are introduced in QA systems. The second objective was to bring systems based on Textual Entailment to the Automatic Hypothesis Generation problem which is not part itself of the Recognising Textual Entailment (RTE) task but a need of the Answer Validation setting. 9 groups have participated with 16 runs in 4 different languages. Compared with the QA systems, the results show an evidence of the potential gain that more sophisticated AV modules introduce in the task of QA.

show abstract

Section: Question Answering Trackmentioning

confidence: 99%

Section: Development Collectionsmentioning

confidence: 99%

See 1 more Smart Citation

Overview of the Answer Validation Exercise 2008

Rodrigo

Peñas

Verdejo

2009

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Multilingual tasks such as IR and Question Answering (QA) have been recognized as an important issue in the on-line information access, as it was revealed in the the Cross-Language Evaluation Forum (CLEF) 2006 [6].…”

Section: Introductionmentioning

confidence: 99%

The Negative Effect of Machine Translation on Cross–Lingual Question Answering

Ferrández

2007

Computational Linguistics and Intelligent Text Processing

View full text Add to dashboard Cite

Abstract. This paper presents a study of the negative effect of Machine Translation (MT) on the precision of Cross-Lingual Question Answering (CL-QA). For this research, a English-Spanish Question Answering (QA) system is used. Also, the sets of 200 official questions from CLEF 2004 and 2006 are used. The CL experimental evaluation using MT reveals that the precision of the system drops around 30% with regard to the monolingual Spanish task. Our main contribution consists on a taxonomy of the identified errors caused by using MT and how the errors can be overcome by using our proposals. An experimental evaluation proves that our approach performs better than MT tools, at the same time contributing to this CL-QA system being ranked first at English-Spanish QA CLEF 2006.

show abstract

“…Last year, a new Romanian-to-English (RO-EN) cross-lingual QA task was organised for the first time within the context of the CLEF campaign [10], and it consisted of retrieving answers to Romanian questions from a collection of English documents. This year's task [6] was similarly organised, with the exception that all questions were clustered in classes related to the same topic, some of which even contain anaphoric references to other questions from the same topic class, or to their answers.…”

Section: Introductionmentioning

confidence: 99%

University of Wolverhampton at CLEF 2007

Puşcaşu

Orăsan

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. This paper reports on the participation of the University of Wolverhampton in the Multiple Language Question Answering (QA@CLEF) track of the CLEF 2007 campaign. We approached the Romanian to English cross-lingual task with a Question Answering (QA) system that processes a question in the source language (i.e. Romanian), translates the identified keywords into the target language (i.e. English), and finally searches for answers in the English document collection. We submitted one run of our system that has achieved an overall accuracy of 14%, and a precision over non-NIL answers of 33.73%. Error analysis revealed that this low performance is mainly due to the lack of a reliable translation methodology from the source in the target language.

show abstract

Overview of the CLEF 2006 Multilingual Question Answering Track

Cited by 44 publications

References 7 publications

Overview of the Answer Validation Exercise 2008

Overview of the Answer Validation Exercise 2008

The Negative Effect of Machine Translation on Cross–Lingual Question Answering

University of Wolverhampton at CLEF 2007

Contact Info

Product

Resources

About