External Sandhi and its Relevance to Syntactic Treebanking

Abstract

External sandhi is a linguistic phenomenon which refers to a set of sound changes that occur at word boundaries. These changes are similar to phonological processes such as assimilation and fusion when they apply at the level of prosody, such as in connected speech. External sandhi formation can be orthographically reflected in some languages. External sandhi formation in such languages, causes the occurrence of forms which are morphologically unanalyzable, thus posing a problem for all kind of NLP applications. In this paper, we discuss the implications that this phenomenon has for the syntactic annotation of sentences in Telugu, an Indian language with agglutinative morphology. We describe in detail, how external sandhi formation in Telugu, if not handled prior to dependency annotation, leads either to loss or misrepresentation of syntactic information in the treebank. This phenomenon, we argue, necessitates the introduction of a sandhi splitting stage in the generic annotation pipeline currently being followed for the treebanking of Indian languages. We identify one type of external sandhi widely occurring in the previous version of the Telugu treebank (version 0.2) and manually split all its instances leading to the development of a new version 0.5. We also conduct an experiment with a statistical parser to empirically verify the usefulness of the changes made to the treebank. Comparing the parsing accuracies obtained on versions 0.2 and 0.5 of the treebank, we observe that splitting even just one type of external sandhi leads to an increase in the overall parsing accuracies.

Analyzing Parser Errors

“…But as long as the linguistic integrity of the analysis is maintained, this will not be a disadvantage-after all, most treebanks are overwhelmingly used to develop high performance automatic parsers. For the Telugu dependency treebank Kolachina et al (2011) manually annotated the phenomenon of external`sandhi'. A transition-based dependency parser trained on this modied treebank performed better than the one trained on the old version.…”

“…We understand that doing this is not always trivial; nevertheless, we have tried to focus only on those errors that we thought are due to lack of robust features or to dicult to learn structures. Our work is certainly not without precedent; research in the dependency parsing literature related to feature optimization (Ambati et al, 2010a), (Seeker and Kuhn, 2011), lexicalization (Eryigit et al, 2008), (Kolachina et al, 2011), use of semantics (Bharati et al, 2008), (Ambati et al, 2009), etc. have tried out dierent types of language specic characteristics and explored ways in which they should be used to inuence the parser performance.…”