Metamorphic testing of named entity recognition systems: A case study

Xu, Yezi; Zhi, Zhou; Zhang, Xiaoxia; Wang, Jing; Jiang, Mingyue

doi:10.1049/sfw2.12058

Cited by 6 publications

(3 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(2) The result of one-hot encoding is a sparse encoding result. When the corpus is relatively large, it will cause the dimension of the constructed vocabulary to be too high, resulting in a waste of storage space and increasing the training burden of the model [15] . Therefore, after the rise of deep learning, this method was eliminated and replaced by distributed representation, which uses characteristic vocabulary representation.…”

Section: Vocabulary Representationmentioning

confidence: 99%

Research on Chinese named entity recognition based on feature fusion

Lin,

Mao,

Zhang

2024

Preprint

View full text Add to dashboard Cite

Its core goal is to extract entities with specific meanings from text, and classify and label them. Recognizing entities in text can provide support for downstream tasks such as knowledge graphs, machine translation, and question answering systems, and therefore has important practical significance and research value. At present, the deep learning technology is used to classify Chinese names, and a satisfactory result is obtained. However, due to the characteristics and complexity of Chinese semantics, the current entity recognition methods of Chinese names still have some problems, such as the entity boundary is difficult to be divided, the structure is flexible, and the entity is easy to be confused. This project intends to start with the extraction of multi-source information of Chinese name entities, build semantic information based on multi-source semantic information, and build a semantic entity model based on feature fusion. Therefore, this paper proposes a method of Chinese name ontology recognition based on semantic analysis. Based on this, this paper conducts research on Chinese named entity recognition technology that uses multiple features for semantic enhancement. First, the research background and current situation of named entity recognition are analyzed and the various basic neural networks widely used in Chinese named entity recognition algorithms and the basic structure of mainstream models are introduced. Secondly, the entity boundaries faced by current Chinese named entity recognition are difficult to divide., flexible word formation, entity ambiguity and other difficulties, two Chinese named entity recognition models based on external semantic information enhancement are proposed to solve the above-mentioned problems existing in the Chinese named entity recognition task and further improve the three different models of MSRA, Resume and Weibo.

show abstract

Section: Vocabulary Representationmentioning

confidence: 99%

Research on Chinese named entity recognition based on feature fusion

Lin,

Mao,

Zhang

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The adversarial attack method needs to access the parameters of the NER models, thus it can not be conveniently adopted to test the commercial NER APIs. Xu et al [37] adopted two specialized metamorphic relations to test an industrial NER system called Litigant. This approach can only be used to test Litigant because it relies on the characteristics (e.g., plaintiff and defendant roles in two companies) and application contexts (e.g., the transformation between parent and sub-company names via extension) of Litigant.…”

Section: Robustness Of Ner Systemsmentioning

confidence: 99%

The Shenzhen-Hong Kong Dialectics

Hu¹

2020

The Shenzhen Phenomenon

View full text Add to dashboard Cite

Recent years have witnessed a rapid development of immersive multimedia which bridges the gap between the real world and virtual space. Volumetric videos, as an emerging representative 3D video paradigm that empowers extended reality, stand out to provide unprecedented immersive and interactive video watching experience. Despite the tremendous potential, the research towards 3D volumetric video is still in its infancy, relying on sufficient and complete datasets for further exploration. However, existing related volumetric video datasets mostly only include a single object, lacking details about the scene and the interaction between them. In this paper, we focus on the current most widely used data format, point cloud, and for the first time release a fullscene volumetric video dataset that includes multiple people and their daily activities interacting with the external environments. Comprehensive dataset description and analysis are conducted, with potential usage of this dataset. The dataset and additional tools can be accessed via the following website: https://cuhkszinml.github.io/full_scene_volumetric_video_dataset/.

show abstract

“…Metamorphic testing (MT) [11], [12], [33] is one of the most popular approaches, alleviating the oracle problem and has been successfully adopted in different domains [45], [8], [13], [5], [17], [19], [35], [40], [41], [43]. Instead of focusing on the correctness of individual outputs, MT examines the relations called metamorphic relations (MRs), among the multiple executions of the SUT.…”

Section: Introductionmentioning

confidence: 99%

Metamorphic Testing of Advanced Driver-Assistance Systems: Implementing Euro NCAP Standards on OpenStreetMap

Iqbal

2023

2023 IEEE/ACM 8th International Workshop on Metamorphic Testing (MET)

View full text Add to dashboard Cite

Simulation testing is considered the supplement to ensure the safety of Autonomous driving (AD) and advanced driving assistance (ADAS's) systems in terms of time and costs. However, it is very difficult and challenging when the simulation results are unexpected. This work presents a simulation-based metamorphic testing (MT) approach to test the ADAS system, implementing the European new car assessment program (Euro NCAP) standards on OpenStreetMap (OSM). We first defined input patterns and relations related to autonomous driving, following the principles of MT. To assess the approach, we executed three tests in two steps at both the design and system levels. Our results show that none of the three (source) tests detected any collisions. However, for follow-up test cases, the ego vehicle failed to apply brakes to avoid a collision when the speed changed. A real-life issue in the system was immediately revealed and confirmed by the development team. We then designed a mechanism and continued the test to check whether the recorded collisions were avoidable. Our results (rate of 5.8%) indicate the fault detection effectiveness when testing the ADAS system. Although we applied the approach to testing the ADAS driving performance, it can be applied to other AD systems. This research, therefore, provides a systematic way to design and test autonomous driving technologies and integrate testing standards with metamorphic testing.

show abstract

Metamorphic testing of named entity recognition systems: A case study

Cited by 6 publications

References 38 publications

Research on Chinese named entity recognition based on feature fusion

Research on Chinese named entity recognition based on feature fusion

The Shenzhen-Hong Kong Dialectics

Metamorphic Testing of Advanced Driver-Assistance Systems: Implementing Euro NCAP Standards on OpenStreetMap

Contact Info

Product

Resources

About