Looking Back to the EVALITA Constituency Parsing Task: 2007-2011

Abstract

Abstract. The aim of the EVALITA Parsing Task (EPT) is at defining and extending Italian state-of-the-art parsing by encouraging the application of existing models and approaches, comparing paradigms and annotation formats. Therefore, in all the editions, held respectively in 2007, 2009 and 2011, the Task has been organized around two tracks, namely Dependency Parsing and Constituency Parsing, exploiting the same data sets made available by the organizers in two different formats.This paper describes the Constituency Parsing Task assuming a historical perspective, but mainly focussing on the last edition held in 2011. It presents and compares the resources exploited for development and testing, the participant systems and the results, showing also how the resources and scores improved during the three editions of this contest.

Tree Kernels-based Discriminative Reranker for Italian Constituency Parsers

“…Then, our TK-based reranker further improved it up to 2 absolute percent points. It should also be noted that our best reranking result is 3.54 absolute points better than the best outcome reported in (Bosco et al, 2013), i.e., 83.27.…”

“…Our parser obtained an LF of 86.28% for sentences with less than 40 words and a score of 85.61% for all sentences. Table 1 also reports the results of the Berkeley parser obtained by Bosco et al (2013). For comparison purposes, we trained our own version of the Berkeley parser.…”

“…We selected such number of cycles applying 10-fold cross validation on the training set. Similarly to Bosco et al (2013), we specialized (ii) the group of features (i.e., either trees or trees + feat.) and (iii) the number, n, of parse trees used to generate the reranker training data.…”

“…In the last EvalIta edition, such gap diminished as the Italian parser labeled F1 increased from 78.73% (EvalIta 2009) to 82.96% (EvalIta 2011). Some years later the parser F1 improved to 83.27% (Bosco et al, 2013). However, the performance of the best English parser (McClosky et al, 2006), i.e., 92.1%, is still far away.…”

“…Consequently, a large body of work has been also devoted to its design for Italian language (Bosco et al, 2007;Bosco et al, 2009;Bosco and Mazzei, 2011). However, the accuracy reported for the best parser is still far behind the state of the art of other languages, e.g., English.…”