How to Cope with Change? - Preserving Validity of Predictive Services over Time

Abstract

Companies more and more rely on predictive services which are constantly monitoring and analyzing the available data streams for better service offerings. However, sudden or incremental changes in those streams are a challenge for the validity and proper functionality of the predictive service over time. We develop a framework which allows to characterize and differentiate predictive services with regard to their ongoing validity. Furthermore, this work proposes a research agenda of worthwhile research topics to improve the long-term validity of predictive services. In our work, we especially focus on different scenarios of true label availability for predictive services as well as the integration of expert knowledge. With these insights at hand, we lay an important foundation for future research in the field of valid predictive services.

What to prioritize? Natural Language Processing for the Development of a Modern Bug Tracking Solution in Hardware Development

“…A limitation to our approach is the data set used for our experiments since the quality of raw data is heterogeneous. Further, we do not account for concept drift [36] in the selection of our training, validation, and test sets. Since our active learning approach did not improve the learning rate, we propose to use other semi-supervised learning methods, or active learning with a different uncertainty measure in order to receive fewer outliers.…”

A2Log: Attentive Augmented Log Anomaly Detection

“…Unsupervised means that the model is trained on normal log messages only. As not all possible anomalies in log data can be known and used for training [34], unsupervised approaches are well suited for log data scenarios, and thus are of high interest for industry and academia [11,3]. Therefore, the challenge is to develop a good understanding of normal log messages, e.g.…”

“…These logging events evolve due to software updates. Therefore unsupervised methods are valuable due to their sense of new and unknown anomalies [3,4]. The second benefit of unsupervised methods is that they do not require labeled data, which is hard to obtain and cost-intensive [5].…”

Unterstützung der Wissensarbeit durch künstliche Intelligenz – Anforderungen an die Gestaltung maschinellen Lernens

“…kalibriert werden. Dazu ist es notwendig, zu erkennen, wann solche Veränderungen eingetreten sind und -in der einschlägigen Terminologie -ein "concept drift" vorliegt (Baier 2019). Dies ist insbesondere dann kritisch, wenn die Assistenz in einer Empfehlung für eine Aktion mündet, die selbst wieder die Datenlage beeinflusst: so wird z.B.…”