Interpretable duplicate question detection models based on attention mechanism

Zhou, Qifeng; Liu, Xiang; Wang, Qing

doi:10.1016/j.ins.2020.07.048

Cited by 24 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Valentine system, (26) for instance, facilitates the organization and execution of largescale matching experiments, which is vital in today's data-rich environment. On the other hand, a dynamic graph framework introduced in (37,40,41,42,43) models the interaction between natural language utterances and database schemas, potentially advancing the handling of evolving schemas. There is still a need for research to perform incremental schema matching on dynamic data.…”

Section: Schema Matching Challengesmentioning

confidence: 99%

Overview on Data Ingestion and Schema Matching

El Haddadi,

Chevalier,

Dousset

et al. 2024

Data and Metadata

View full text Add to dashboard Cite

This overview traced the evolution of data management, transitioning from traditional ETL processes to addressing contemporary challenges in Big Data, with a particular emphasis on data ingestion and schema matching. It explored the classification of data ingestion into batch, real-time, and hybrid processing, underscoring the challenges associated with data quality and heterogeneity. Central to the discussion was the role of schema mapping in data alignment, proving indispensable for linking diverse data sources. Recent advancements, notably the adoption of machine learning techniques, were significantly reshaping the landscape. The paper also addressed current challenges, including the integration of new technologies and the necessity for effective schema matching solutions, highlighting the continuously evolving nature of schema matching in the context of Big Data.

show abstract

Section: Schema Matching Challengesmentioning

confidence: 99%

Overview on Data Ingestion and Schema Matching

El Haddadi,

Chevalier,

Dousset

et al. 2024

Data and Metadata

View full text Add to dashboard Cite

show abstract

“…This is why interpretable task-specific solutions are also relevant. In [53], the authors focused their attention on explaining the duplicate question detection task developing a specific model based on the attention mechanism, proposing to interpret the model results by visually analyzing their attention matrix to understand the inter-words relations learned by the model. However, exploiting attention can be performed only for black-box models that are based on this mechanism, and it can be hard to interpret for non-expert users.…”

Section: Task-specific Approachesmentioning

confidence: 99%

Trusting deep learning natural-language models via local and global explanations

et al. 2022

View full text Add to dashboard Cite

Despite the high accuracy offered by state-of-the-art deep natural-language models (e.g., LSTM, BERT), their application in real-life settings is still widely limited, as they behave like a black-box to the end-user. Hence, explainability is rapidly becoming a fundamental requirement of future-generation data-driven systems based on deep-learning approaches. Several attempts to fulfill the existing gap between accuracy and interpretability have been made. However, robust and specialized eXplainable Artificial Intelligence solutions, tailored to deep natural-language models, are still missing. We propose a new framework, named T-EBAnO, which provides innovative prediction-local and class-based model-global explanation strategies tailored to deep learning natural-language models. Given a deep NLP model and the textual input data, T-EBAnO provides an objective, human-readable, domain-specific assessment of the reasons behind the automatic decision-making process. Specifically, the framework extracts sets of interpretable features mining the inner knowledge of the model. Then, it quantifies the influence of each feature during the prediction process by exploiting the normalized Perturbation Influence Relation index at the local level and the novel Global Absolute Influence and Global Relative Influence indexes at the global level. The effectiveness and the quality of the local and global explanations obtained with T-EBAnO are proved on an extensive set of experiments addressing different tasks, such as a sentiment-analysis task performed by a fine-tuned BERT model and a toxic-comment classification task performed by an LSTM model. The quality of the explanations proposed by T-EBAnO, and, specifically, the correlation between the influence index and human judgment, has been evaluated by humans in a survey with more than 4000 judgments. To prove the generality of T-EBAnO and its model/task-independent methodology, experiments with other models (ALBERT, ULMFit) on popular public datasets (Ag News and Cola) are also discussed in detail.

show abstract

“…This is why interpretable task-specific solutions are also relevant. In [28] the authors focused their attention on explaining the duplicate question detection task developing a specific model based on the attention mechanism, proposing to interpret the model results by visually analyzing their attention matrix to understand the inter-words relations learned by the model. However, exploiting attention can be performed only for black-box models that are based on this mechanism, and it can be hard to interpret for non-expert users.…”

Section: Literature Reviewmentioning

confidence: 99%

Explaining the Deep Natural Language Processing by Mining Textual Interpretable Features

Ventura¹,

Greco²,

Apiletti³

et al. 2021

Preprint

View full text Add to dashboard Cite

Despite the high accuracy offered by state-of-the-art deep natural-language models (e.g. LSTM, BERT), their application in reallife settings is still widely limited, as they behave like a black-box to the end-user. Hence, explainability is rapidly becoming a fundamental requirement of future-generation data-driven systems based on deep-learning approaches. Several attempts to fulfill the existing gap between accuracy and interpretability have been done. However, robust and specialized xAI (Explainable Artificial Intelligence) solutions tailored to deep natural-language models are still missing. We propose a new framework, named T-EBAnO, which provides innovative prediction-local and class-based model-global explanation strategies tailored to black-box deep natural-language models. Given a deep NLP model and the textual input data, T-EBAnO provides an objective, human-readable, domain-specific assessment of the reasons behind the automatic decision-making process. Specifically, the framework extracts sets of interpretable features mining the inner knowledge of the model. Then, it quantifies the influence of each feature during the prediction process by exploiting the novel normalized Perturbation Influence Relation index at the local level and the novel Global Absolute Influence and Global Relative Influence indexes at the global level. The effectiveness and the quality of the local and global explanations obtained with T-EBAnO are proved on (i) a sentiment analysis task performed by a fine-tuned BERT model, and (ii) a toxic comment classification task performed by an LSTM model.

show abstract

Interpretable duplicate question detection models based on attention mechanism

Cited by 24 publications

References 7 publications

Overview on Data Ingestion and Schema Matching

Overview on Data Ingestion and Schema Matching

Trusting deep learning natural-language models via local and global explanations

Explaining the Deep Natural Language Processing by Mining Textual Interpretable Features

Contact Info

Product

Resources

About