Benchmarking applied semantic inference: The PASCAL recognising textual entailment challenges

Identifying that the same meaning is expressed by, or can be inferred from, various language expressions is a major challenge for natural language understanding applications such as information extraction, question answering and automatic summarization. Dagan and Glickman [5] proposed Textual Entailment, the task of deciding whether a target text follows from a source text, as a unifying framework for modeling language variability, which has often been addressed in an applicationspecific manner. In this paper we describe the series of benchmarks developed for the textual entailment recognition task, known as the PASCAL RTE Challenges. As a concrete example, we describe in detail the second RTE challenge, in which our methodology was consolidated, and served as a basis for the subsequent RTE challenges. The impressive success of these challenges established textual entailment as an active research area in natural language processing, attracting a growing community of researchers.