Inductive transfer for learning Bayesian networks

Abstract

In several domains it is common to have data from different, but closely related problems. For instance, in manufacturing, many products follow the same industrial process but with different conditions; or in industrial diagnosis, where there is equipment with similar specifications. In these cases it is common to have plenty of data for some scenarios but very little for others. In order to learn accurate models for rare cases, it is desirable to use data and knowledge from similar cases; a technique known as transfer learning. In this paper we propose an inductive transfer learning method for Bayesian networks, that considers both structure and parameter learning. For structure learning we use conditional independence tests, by combining measures from the target task with those obtained from one or more auxiliary tasks, using a novel weighted sum of the conditional independence measures. For parameter learning, we propose two variants of the linear pool for probability aggregation, combining the probability estimates from the target task with those from the auxiliary tasks. To validate our approach, we used three Bayesian networks models that are commonly used for evaluating learning techniques, and generated variants of each model by changing the structure as well as the parameters. We then learned one of the variants with a small dataset and combined it with information from the other variants. The experimental results show a significant improvement in terms of structure and parameters when we transfer knowledge from similar tasks. We also evaluated the method with real-world data from a manufacturing process considering several products, obtaining an improvement in terms of log-likelihood between the data and the model when we do transfer learning from related products.

Robust representation of shape in a Grey parrot (Psittacus erithacus)

“…Notably, nonhuman subjects in these paradigms may fail when given strong transfer tests (Peissig et al, 2005). Strong generalization, the ability to perform under the widest circumstances without additional training, provides a proven evaluative metric for this broader knowledge in the fields of education (Gober, Baker, & Sao Pedro, 2011) and even machine learning (e.g., Luis, Sucar & Morales, 2010). Thus, a wider demonstration of this phenomenon would solidify claims for the robustness of nonhuman abilities with respect to learning and cognition.…”

Multi-source transfer learning of time series in cyclical manufacturing

“…Successful application in chemistry-oriented manufacturing processes with the usage of chemometric modeling techniques are presented in Nikzad-Langerodi et al (2018), Malli et al (2017). Another successful application of transfer learning in intelligent manufacturing for improving product quality was presented in Luis et al (2010).…”

On the empirical justification of theoretical heuristic transference and learning

“…Transfer learning, with the support of CI, is usually turned towards domains such as neural networks, fuzzy systems, and evolutionary computation. It is applied in real-world applications to: natural language processing (Huang et al 2013;Swietojanski et al 2012;Behbood et al 2013b), computer vision (Cireşan et al 2012;Kandaswamy et al 2014;Shell and Coupland 2012), biology (Celiberto et al 2011;Niculescu-Mizil and Caruana 2007;Oyen and Lane 2013), finance (Behbood et al 2011(Behbood et al , 2013a(Behbood et al , 2014, and business management (Ma et al 2012;Luis et al 2010;Shell 2013).…”