Designing Explainable Text Classification Pipelines: Insights from IT Ticket Complexity Prediction Case Study

Revina, Aleksandra; Búza, Krisztián; Meister, Vera G.

doi:10.1007/978-3-030-64949-4_10

Cited by 6 publications

(9 citation statements)

References 108 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, we performed a comparative analysis with the current benchmark from the original study [16]. Our proposed architecture achieved 68% AUC for SVM and 43% for NB compared to the benchmark AUC score of 50% (for NB) and 41% (for SVM) reported in the original study.…”

Section: B Results For Datasetmentioning

confidence: 93%

See 1 more Smart Citation

Multiple Severity-Level Classifications for IT Incident Risk Prediction

Ahmed¹,

Singh²,

Doherty³

et al. 2022

2022 9th International Conference on Soft Computing &Amp; Machine Intelligence (ISCMI)

View full text Add to dashboard Cite

The adoption of Artificial Intelligence (AI) is now widespread in Information Technology (IT) support. A particular area of interest is in the automation of IT incident management (i.e., the handling of an unusual event that hampers the quality of IT services in the most optimized manner). In this paper, we propose a framework using state-of-art algorithms to classify and predict the severity of such incidents (commonly labeled as High, Medium, and Low severity). We argue that the proposed framework would accelerate the process of handling IT incidents with improved accuracy. The experimentation was performed on the IT Service Management (ITSM) dataset containing 500,000 real-time incident descriptions with their encoded labels (Dataset 1) from a reputable IT firm. Our results showed that the Transformer models outperformed machine learning (ML) and other deep learning (DL) models with a 98% AUC score to predict the three severity classes. We tested our framework with an open-access dataset (Dataset 2) to further validate our findings. Our framework produced a 44% improvement in AUC score compared to the existing benchmark approaches. The results show the plausibility of AI algorithms in automating the prioritization of incident processing in large IT systems.

show abstract

Section: B Results For Datasetmentioning

confidence: 93%

“…In order to validate the performance of our framework, we selected an open source IT incident dataset [16] impact. For compatibility purpose, we have combined the title and body columns into a single text column and replaced the labels of severity (the urgency column) to High, Medium and Low respectively.…”

Section: Methodsmentioning

confidence: 99%

Multiple Severity-Level Classifications for IT Incident Risk Prediction

Ahmed¹,

Singh²,

Doherty³

et al. 2022

2022 9th International Conference on Soft Computing &Amp; Machine Intelligence (ISCMI)

View full text Add to dashboard Cite

show abstract

“…In contrast, the proposed system can be used for user generated ticket as well as for system generated tickets. Research in [10,21] use text mining tools to identify different related ticket resolutions from the ticketing database, although the quality of ticket resolution is not taken into account by those systems. Mukunthan et al 6 proposed a multilevel Petri net-based ticket assignment scheme to reduce IT workload of enterprize organization.…”

Section: Related Workmentioning

confidence: 99%

A multiapproach generalized framework for automated solution suggestion of support tickets

Zaidi

Fraz

Shahzad

et al. 2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

Nowadays, customer support systems are one of the key factors in maintaining any big company's reputation and success. These systems are capable of handling a large number of tickets systemically and provides a mechanism to track/logs the communication between customer and support agents. Companies invest huge amounts of money in training support agents and deploying customer care services for their products and services. Support agents are responsible for handling different customer queries and implementing required actions to solve a particular issue or problem raised by the service/product user. In a bigger picture, customer support systems could receive a large amount of ticket raised depending upon the number of users and services being offered. Customer care service gets directly affected due to the high volume of tickets and a limited number of support agents. Therefore, providing support agents with the recommendations about the possible resolution actions for a new ticket would be helpful and can save a lot of time. This study is focused on the development of an end‐to‐end framework for suggesting resolution actions rather than recommending free form resolution text against a newly raised ticket. To develop such a system, the pipeline is broadly divided into four components that are data preprocessing, actions extractor, resolution predictor, and evaluation. In actions extractor module, we have proposed a technique to identify and extract actionable phrases from resolution text. For resolution predictor, we have proposed two different pipelines that are referred as “Similarity Search Model” and “End‐to‐End Model.” The similarity search method is based on a ticket similarity search to find the most relevant historical tickets which then leads to corresponding resolution actions. On the other hand, end‐to‐end model make use of actions extractor module directly and implemented in a way to directly predict resolution actions. To compare and evaluate the mentioned methods on the same ground, we also proposed an actions evaluation criterion which uses BertScore and METEOR score jointly to compute the score against actual and predicted actions for a particular test ticket. The analysis and experiments are performed on the real‐world IBM ticket data set. Overall, we observed that end‐to‐end model outperformed similarity search‐based methods and achieved better performance and scores comparatively. The trained models and code are available at https://bit.ly/2GbUBVk.

show abstract

“…An IT service with an organization is responsible for providing quality service to their customers, and the most challenging aspect of such service is the maintenance of the information system to ensure it remains in a stable condition continuously [1]. Any activity that is not part of the standard operation interferes or reduces the quality of the service (commonly known as incidents) [2]. Incident management is the most critical aspect of IT service management.…”

Section: Introductionmentioning

confidence: 99%

“…Thus far, there has been limited implementation of AIOPS in the domain of IT incidents management [2]. We believe that it is essential to take stock of existing research; accordingly, we formulate a systematic review strategy with a comprehensive approach addressing the research questions as presented in Table : I.…”

Section: Introductionmentioning

confidence: 99%

AI for Information Technology Operation (AIOps): A Review of IT Incident Risk Prediction

Ahmed¹,

Singh²,

Doherty³

et al. 2022

2022 9th International Conference on Soft Computing &Amp; Machine Intelligence (ISCMI)

View full text Add to dashboard Cite

The advancement of Artificial Intelligence has led to a surge in its application in Information Technology (IT) Operations, often termed Artificial Intelligence for IT Operations (AIOPS). One of the most challenging problems in AIOPS is IT Service Management (ITSM), which deals with incidents and anomalies of users, often referred to as tickets. These tickets are resolved by the IT firm support system, which plays a significant role in the company's user experiences, productivity, and profit. Recent advances have been made to automate the prediction of IT incidents and resolve them in a minimal time, utilizing AI models. In this paper, we take stock of the work in this domain and review the challenges. We also highlight the open topics that require further investigation for the advancement of the field.

show abstract

Designing Explainable Text Classification Pipelines: Insights from IT Ticket Complexity Prediction Case Study

Cited by 6 publications

References 108 publications

Multiple Severity-Level Classifications for IT Incident Risk Prediction

Multiple Severity-Level Classifications for IT Incident Risk Prediction

A multiapproach generalized framework for automated solution suggestion of support tickets

AI for Information Technology Operation (AIOps): A Review of IT Incident Risk Prediction

Contact Info

Product

Resources

About