About Time: Do Transformers Learn Temporal Verbal Aspect?

Metheniti, Eleni; Cruys, Tim Van de; Hathout, Nabil

doi:10.18653/v1/2022.cmcl-1.10

Cited by 5 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the location in The boy was fishing at the lake is more salient than in The boy had fished at the lake, as the event is represented as still ongoing). The work by Metheniti et al (2022) focused again on the BERT model and on the aspectual features of telicity and duration. Their setup included a classification task in English and French, and their results proved that in both languages BERT was adequately capturing information on telicity and duration, even in the non-finetuned forms, although it also showed some bias to verb tense and word order.…”

Section: Limitationsmentioning

confidence: 99%

Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP

2023

View full text Add to dashboard Cite

As researchers achieve unprecedented technological breakthroughs in natural language processing, the need to understand the systems underlying these advances is more pertinent than ever. BlackboxNLP, now in its sixth iteration, has played an important role in bringing together scholars from a diverse range of backgrounds in order to rigorously study the behavior, representations, and computations of "black-box" neural network models. Our workshop showcases original, cutting-edge research on topics including but not limited to:• analysis of representations via probing and related techniques;• explanation methods such as feature attribution, free-text explanations, or structured explanations; • interdisciplinary methods (e.g., from neuroscience, cognitive science, computer vision, etc.); • interpretable architectures and neural network modules;• mechanistic interpretability and reverse engineering of neural computations; • open-source tools for analysis, visualization, and/or explanation; • opinions about the state of interpretability and explainable NLP; and • targeted evaluations using simplified or formal languages.The sixth BlackboxNLP workshop will be held in Singapore on December 7, 2023, hosted by the Conference on Empirical Methods in Natural Language Processing (EMNLP). 29 full papers and 17 non-archival extended abstracts were accepted for in-person and online presentations, from a total of 66 submissions. This year's workshop will also feature 19 papers on interpretability from the Findings of the ACL: EMNLP 2023, as well as two invited talks and a panel discussion with experts in the field.BlackboxNLP 2023 would not have been possible without the high-quality peer reviews submitted by our program committee, as well as the logistical assistance provided by the EMNLP organizing committee. We gratefully acknowledge financial support from our sponsors, Google and Apple. Our invited speakers, panelists, authors, and presenters have allowed us to put together an outstanding program for all participants to enjoy.Welcome to BlackboxNLP! We look forward to seeing you in Singapore and online.

show abstract

Section: Limitationsmentioning

confidence: 99%

Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP

2023

View full text Add to dashboard Cite

show abstract

“…They show that this is useful for picking the correct tense in French translations of the English Simple Past. Several more recent studies have shown that distributional and neural models can be trained to predict telicity as annotated in available datasets (Kober et al, 2020;Metheniti et al, 2021;Metheniti, 2022). BERT-style models perform well on existing telicity datasets (with larger models outperforming smaller models), yet it is still unclear how or whether they actually capture aspect.…”

Section: Telicitymentioning

confidence: 99%

“…Lexical and grammatical aspect play essential roles in semantic interpretation (Smith, 2003), and yet even state-of-the-art natural language understanding (NLU) systems do not address these linguistic phenomena systematically (Metheniti, 2022). Consider this example: an NLU-based personal assistant, noticing the boarding time of a flight, tells a passenger (who is still shopping at the airport) "You miss flights" (i.e.…”

Section: Introductionmentioning

confidence: 99%

“…While some NLP systems trained on end-to-end datasets may get some of these cases right, we are not aware of a system treating such problems in a principled manner. A recent study finds that transformer-based embeddings are useful for classifying clausal aspect (Metheniti, 2022), but it remains unclear just how these models learn or incorporate aspectual information. This survey focuses on modeling aspectual information as it is annotated in the datasets listed in Table 1.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Kind Introduction to Lexical and Grammatical Aspect, with a Survey of Computational Approaches

Friedrich,

Xue,

Palmer

2023

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

View full text Add to dashboard Cite

Aspectual meaning refers to how the internal temporal structure of situations is presented. This includes whether a situation is described as a state or as an event, whether the situation is finished or ongoing, and whether it is viewed as a whole or with a focus on a particular phase. This survey gives an overview of computational approaches to modeling lexical and grammatical aspect along with intuitive explanations of the necessary linguistic concepts and terminology. In particular, we describe the concepts of stativity, telicity, habituality, perfective and imperfective, as well as influential inventories of eventuality and situation types. Aspect is a crucial component of semantics, especially for precise reporting of the temporal structure of situations, and future NLP approaches need to be able to handle and evaluate it systematically.

show abstract

“…Our use of distributional semantic representations is furthermore motivated by the fact that they are readily available in numerous languages, and that they, contrary to manually constructed lexicons such as VerbNet (Schuler and Palmer, 2005) or LCS (Dorr and Olsen, 1997), scale well with growing amounts of data and across different languages. Furthermore, there is a growing body of evidence that models based on the distributional hypothesis capture some facets of aspect (Kober et al, 2020;Metheniti et al, 2022), despite the fact that aspect is represented in a very diverse manner across languages.…”

Section: Computational Experimentsmentioning

confidence: 99%

Proceedings of the 15th International Conference on Natural Language Generation

2022

View full text Add to dashboard Cite

Bio: Margaret Mitchell is a researcher working on Ethical AI, currently focused on the ins and outs of ethics-informed AI development in tech. She has published over 50 papers on natural language generation, assistive technology, computer vision, and AI ethics, and holds multiple patents in the areas of conversation generation and sentiment classification. She currently works at Hugging Face driving forward work in the ML development ecosystem, ML data governance, AI evaluation, and AI ethics. She previously worked at Google AI as a Staff Research Scientist, where she founded and co-led Google's Ethical AI group, focused on foundational AI ethics research and operationalizing AI ethics Googleinternally. Before joining Google, she was a researcher at Microsoft Research, focused on computer vision-to-language generation; and was a postdoc at Johns Hopkins, focused on Bayesian modeling and information extraction. She holds a PhD in Computer Science from the University of Aberdeen and a Master's in computational linguistics from the University of Washington. While earning her degrees, she also worked from 2005-2012 on machine learning, neurological disorders, and assistive technology at Oregon Health and Science University. She has spearheaded a number of workshops and initiatives at the intersections of diversity, inclusion, computer science, and ethics. Her work has received awards from Secretary of Defense Ash Carter and the American Foundation for the Blind, and has been implemented by multiple technology companies. She likes gardening, dogs, and cats.

show abstract

About Time: Do Transformers Learn Temporal Verbal Aspect?

Cited by 5 publications

References 24 publications

Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP

Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP

A Kind Introduction to Lexical and Grammatical Aspect, with a Survey of Computational Approaches

Proceedings of the 15th International Conference on Natural Language Generation

Contact Info

Product

Resources

About