A POS Tagger for Social Media Texts Trained on Web Comments

Neunerdt, Melanie; Reyer, Michael; Mathar, Rudolf

doi:10.17562/pb-48-8

Cited by 11 publications

(11 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These new challenges include the management of metainformation included in the text (for example as tags in tweets) (Bouillot et al, 2013), the detection of typos and unconventional spelling, word short- enings (Neunerdt et al, 2013;Moreira et al, 2013) and slang and emoticons (Balahur, 2013), among others. Another challenge that should be taken into account is that while clinical records and medical literature can be mapped to terminological resources or biomedical ontologies, lay terminology used by patients to describe their treatments and their effects, in general, is not collected in any terminological resource, which would facilitate the automatic processing of this kind of texts.…”

Section: Resultsmentioning

confidence: 99%

Proceedings of BioNLP 2014

2014

View full text Add to dashboard Cite

IntroductionThe first day of the BioNLP 2014 workshop continues following the course set by the first ACL workshop on Natural Language Processing in the Biomedical Domain that was held in 2002: BioNLP 2014 provides a venue for exploring challenges and techniques in processing biomedical language and brings together researchers from computational linguistics and biomedical informatics. The submissions to the first day of 2014 workshop organized by SIGBioMed were traditionally very strong and continued demonstrating the considerable breadth of research in biomedical language processing. The 2014 workshop has accepted 12 full and short papers for oral presentations and 7 posters. The first day of the workshop features a keynote that expands the scope of BioNLP beyond its already remarkable breadth The second day of the workshop features a paper submitted to the special track on NLP approaches for assessment of clinical conditions. Kathleen C. Fraser presents the featured talk on using statistical parsing to detect agrammatic aphasia. The track organizers, Thamar Solorio and Yang Liu, serve as discussants.The second day further features an exciting panel that brings together organizers of several shared tasks in biomedical information retrieval and natural language processing. The panel introduces the workshop participants to the long-standing and relatively new community-wide challenges in biomedical and clinical language processing. It also provides an opportunity to discuss the future of the shared tasks in this domain. AcknowledgmentsAs always, we are profoundly grateful to the authors who chose BioNLP as venue for presenting their innovative research. The authors' willingness to share their work through BioNLP consistently makes the workshop noteworthy among the increasing numbers of available venues. We are equally indebted to the program committee members (listed elsewhere in this volume) who produced at least three thorough reviews per paper on a tight review schedule and with an admirable level of insight.iii AbstractZoonotic viruses, viruses that are transmittable between animals and humans, represent emerging or re-emerging pathogens that pose significant public health threats throughout the world. It is therefore crucial to advance current surveillance mechanisms for these viruses through outlets such as phylogeography. Phylogeographic techniques may be applied to trace the origins and geographical distribution of these viruses using sequence and location data, which are often obtained from publicly available databases such as GenBank. Despite the abundance of zoonotic viral sequence data in GenBank records, phylogeographic analysis of these viruses is greatly limited by the lack of adequate geographic metadata. Although more detailed information may often be found in the related articles referenced in these records, manual extraction of this information presents a severe bottleneck. In this work, we propose an automated system for extracting this information using Natural Language Processing (NLP)...

show abstract

Section: Resultsmentioning

confidence: 99%

Proceedings of BioNLP 2014

2014

View full text Add to dashboard Cite

show abstract

“…As such, they can be compared to other types of non-standard language, including computer-mediated communication or learner texts. Previous studies in this area have noted the challenges of applying the existing NLP tools and tagsets, which are often trained on the basis of newspaper language, to the data that deviates from this standard (Neunerdt et al, 2013;Zinsmeister et al, 2014). This issue is addressed by development of modified taggers as well as adaptations of the tagsets, for instance to include tags that are unique to a certain type of data (cf.…”

Section: Part Of Speech Annotationmentioning

confidence: 99%

“…e.g. Neunerdt et al, 2013 for annotation of social media texts or Westpfahl and Schmidt, 2013 for enrichment of spoken German).…”

Section: Part Of Speech Annotationmentioning

confidence: 99%

Part of Speech Annotation of Intermediate Versions in the Keystroke Logged Translation Corpus

Serbina

Niemietz

Fricke³

et al. 2015

Proceedings of the 9th Linguistic Annotation Workshop

View full text Add to dashboard Cite

Translation process data contains noncanonical features such as incomplete word tokens, non-sequential string modifications and syntactically deficient structures. While these features are often removed for the final translation product, they are present in the unfolding text (i.e. intermediate translation versions). This paper describes tools developed to semi-automatically process intermediate versions of translation data to facilitate quantitative analysis of linguistic means employed in translation strategies. We examine the data from a translation experiment with the help of these tools.

show abstract

“…We used several instruments in this experiment, Moses [1] as machine translators, SRILM [26] to building language and PoS models, Giza++ [27] for word alignment process, and Grammar Postagger for PoS tagging. Furthermore, we use the BLEU method [28] for scoring the translation results.…”

Section: Experiments On Smtmentioning

confidence: 99%

“…Mart [23] used 47 tags to build a Spanish treebank in Spanish. For developed part-of-speech tagger, Avontuur et al [24] used 25 tags for Dutch, Singha et al [25] used 97 tags for Manipuri, Neunerdt et al [26] used 54 tags for German.…”

Section: Introductionmentioning

confidence: 99%

A Novel Part-of-Speech Set Developing Method for Statistical Machine Translation

Sujaini¹,

Kuspriyanto²,

Arman³

et al. 2014

TELKOMNIKA

View full text Add to dashboard Cite

Part of speech (PoS) IntroductionThe dream of automatically translating documents between two languages is one of the oldest pursuits of artificial intelligence research. Now, armed with vast amounts of example translations and powerful computers, we can witness significant progress toward achieving that dream. Statistical analysis of bilingual parallel corpora allow for the automatic construction of machine translation systems. Already, for some language pairs, statistical systems are the best machine translation systems currently available.Statistical Machine Translation is corpus-based and consequently requires a parallel corpus to learn a model [1], [2]. Parallel corpora are different from normal text corpora in that they are not just a collection of texts, but are bilingual or multilingual and structured so that every sentence is linked to its translations.Some works have shown that the translation quality can be increased by using additional features such as lemma, part of speech (PoS), gender and others. In their research, Koehn and Hoang [3] explained that by adding a factor of part-of-speech in English-German translator system, the quality of the translation was increased from 18.04% to 18.15%. They also showed that by using morphological factors and part-of-speech, the English-Spanish translator system quality was increased from 23.41% to 24.25%.Youssef et al.[4] examined the factors on adding part-of-speech on statistical translation system for English-Arabic. Research results showed that the addition of a factor of part-of-speech can improve the quality of translation from 0.6095% to 0.6394%. Razavian and Vogel [5] examined the factors on adding to the statistics based interpreter systems, for EnglishIraqi interpreter system, the quality of the translation was improved from 15.62% to 16.41%; for the Spanish-English translator system, the quality of the translation was improved from 32.53% to 32.84%; and for Arabic-English translator system, the quality of the translation was improved from 41.70% to 42.74%.For English-Indonesian, Sujaini et al.[6] conducted a study of the addition of PoS factors based on a statistical translator system factors. The results of these studies indicated that the PoS factor increased the quality of the English-Indonesian translation of 2%, from 31.26% to 33.26%.Grammatically, words can be divided into two categories: open class and closed class. Open class is a class category which number of words always increases over time, while closed class is a class category whose words are fixed. Grammatically different categories of words, commonly called Part of Speech [1].

show abstract

A POS Tagger for Social Media Texts Trained on Web Comments

Cited by 11 publications

References 15 publications

Proceedings of BioNLP 2014

Proceedings of BioNLP 2014

Part of Speech Annotation of Intermediate Versions in the Keystroke Logged Translation Corpus

A Novel Part-of-Speech Set Developing Method for Statistical Machine Translation

Contact Info

Product

Resources

About