ProcessTransformer: Predictive Business Process Monitoring with Transformer Network

Abstract

Predictive business process monitoring focuses on predicting future characteristics of a running process using event logs. The foresight into process execution promises great potentials for efficient operations, better resource management, and effective customer services. Deep learning-based approaches have been widely adopted in process mining to address the limitations of classical algorithms for solving multiple problems, especially the next event and remaining-time prediction tasks. Nevertheless, designing a deep neural architecture that performs competitively across various tasks is challenging as existing methods fail to capture long-range dependencies in the input sequences and perform poorly for lengthy process traces. In this paper, we propose ProcessTransformer, an approach for learning high-level representations from event logs with an attention-based network. Our model incorporates long-range memory and relies on a self-attention mechanism to establish dependencies between a multitude of event sequences and corresponding outputs. We evaluate the applicability of our technique on nine real event logs. We demonstrate that the transformer-based model outperforms several baselines of prior techniques by obtaining on average above 80% accuracy for the task of predicting the next activity. Our method also perform competitively, compared to baselines, for the tasks of predicting event time and remaining time of a running case.

MTLFormer: Multi-Task Learning Guided Transformer Network for Business Process Prediction

“…The model can be applied to long-distance dependent feature learning and concurrent execution of multiple tasks in the data and can well solve the problems of LSTM in training models, thus improving the model's prediction performance. On this basis, Bukhsh et al [11] proposed the ProcessTransformer model, i.e., modifying the Transformer structure according to the specific process prediction task, to achieve the desired prediction effect.…”

“…Based on these mainstream process prediction methods based on RNN and LSTM networks, some studies still use the attention mechanism as an optimization strategy to improve the performance of prediction models. For example, Bukhsh et al [11] proposed the Process Transformer model, that is, to modify the structure of the Transformer network according to specific process prediction tasks to achieve ideal prediction results. Similarly, Wickramanayake et al [28] proposed two types of attention for the prediction task of future activities: the event-level attention to capturing the impact of specific events on the prediction task and the attribute-level attention to reveal which attributes of events affect the prediction task.…”

Conformance-oriented Predictive Process Monitoring in BPaaS Based on Combination of Neural Networks

“…The existing vast majority of PPM techniques can be grouped into three categories according to prediction content, including time prediction [17,40,41], outcome prediction [18,[42][43][44][45], and the next activity (sequences) prediction [19,21,41,[46][47][48]. Moreover, all approaches mentioned above can be grouped into three categories depending on the used techniques, including those based on an extended process model, traditional machine learning, and deep learning.…”

A Road-map for Mining Business Process Models via Artificial Intelligence Techniques

“…On the basis of GPT-2 that can create a context processed representation by according attention to different parts of the input sentence, the POP-ON model adds a linear network which exploits the event attribute for the residual structure within the Transformer Decoder Layer to reflect elements in the prediction. In this case, the article 7 [80] offers another approach known as the transformer process to solve the problems of changing and complex time-specific data sequences because of multiple control flows in real-life event logs. The main contribution is a process transformer technology for learning high-level representations, which can be utilized to reason through a long sequence of networks like as LSTM from sequence event log data with minimal preprocessing step.…”