Many systems are partially stochastic in nature. We have derived datadriven approaches for extracting stochastic state machines (Markov models) directly from observed data. This chapter provides an overview of our approach with numerous practical applications. We have used this approach for inferring shipping patterns, exploiting computer system side-channel information, and detecting botnet activities. For contrast, we include a related data-driven statistical inferencing approach that detects and localizes radiation sources.
IntroductionMarkov models have been widely used for detecting patterns [33,36,7,4,24,25,15,17]. The premise behind a Markov models is that the current state only depends on the previous state and that transition probabilities are stationary. This makes Markov models versatile, as this is a direct result of the causal world we live in. Often,