JARVIS: Joining Adversarial Training With Vision Transformers in Next-Activity Prediction

Abstract

In this paper, we propose a novel predictive process monitoring approach, named JARVIS, that is designed to achieve a balance between accuracy and explainability in the task of next-activity prediction. To this aim, JARVIS represents different process executions (traces) as patches of an image and uses this patch-based representation within a multi-view learning scheme combined with Vision Transformers (ViTs). Using multi-view learning we guarantee good accuracy by leveraging the variety of information recorded in event logs as different patches of an image. The use of ViTs enables the integration of explainable elements directly into the framework of a predictive process model trained to forecast the next trace activity from the completed events in a running trace by utilizing self-attention modules that give paired attention values between two picture patches. Attention modules disclose explainable information concerning views of the business process and events of the trace that influenced the prediction. In addition, we explore the effect of ViT adversarial training to mitigate overfitting and improve the accuracy and robustness of predictive process monitoring. Experiments with various benchmark event logs prove the accuracy of JARVIS compared to several current state-of-the-art methods and draw insights from explanations recovered through the attention modules.

The Explainability of Transformers: Current Status and Directions

“…Cremer et al tested an architecture on 3 datasets for drug toxicity classification [79]. A variation of Attention Rollout was employed by Pasquadibisceglie et al to generate heatmaps in a framework for next-activity prediction in process monitoring [80]. Attention Rollout was employed in conjunction with Grad-CAM (see Section 5.3) by Neto et al to detect metaplasia in upper gastrointestinal endoscopy [81].…”