Scalable Microservice Forensics and Stability Assessment Using Variational Autoencoders

Abstract

We present a deep-learning-based approach to containerized application runtime stability analysis, and an intelligent publishing algorithm that can dynamically adjust the depth of process-level forensics published to a backend incident analysis repository. The approach applies variational autoencoders (VAEs) to learn the stable runtime patterns of container images, and then instantiates these container-specific VAEs to implement stability detection and adaptive forensics publishing. In performance comparisons using a 50-instance container workload, a VAEoptimized service versus a conventional eBPF-based forensic publisher demonstrates 2 orders of magnitude (OM) CPU performance improvement, a 3 OM reduction in network transport volume, and a 4 OM reduction in Elasticsearch storage costs. We evaluate the VAE-based stability detection technique against two attacks, CPUMiner and HTTP-flood attack, finding that it is effective in isolating both anomalies. We believe this technique provides a novel approach to integrating fine-grained process monitoring and digital-forensic services into large container ecosystems that today simply cannot be monitored by conventional techniques.

A Survey on Forensics and Compliance Auditing for Critical Infrastructure Protection

“…However, the topic of forensic investigation in containerized environments is a complex task raising new challenges [120], due to the fact that instances can be started and stopped on different systems, which results in an ongoing change in the structure of the network, as well as their shorter life span which implies that container instances may not be available anymore when a investigation process is triggered. Which such environments in mind, Sharma et al [171] presented a deep learning approach for containerized application runtime stability analysis, and an intelligent publishing algorithm that can dynamically adjust the depth of process-level forensics published to a backend incident analysis repository. Stelly and Roussev [172] presented a scalable containerized framework for forensic computations.…”