Algorithm Selection Using Edge ML and Case-based Reasoning

Abstract

In practical data mining applications area, classification and predictions are the most commonly and frequently used applications which involves intelligent classification methods and algorithms. In machine learning, a large number of classification algorithms are available which take the responsibility of classifying unforeseen data in classification problems and predicts future outcomes. However, the task of selecting a right classification algorithm to produce best results is a challenging issue both for machine learning practitioners and machine learning experts. The reason is the inherit characteristics of classification problems and unpredicted behaviors of the classifiers on these problems. Furthermore, building intelligent ML algorithm recommender over large and complex datasets by extracting their meta-features using conventional computational framework is computationally and timely expensive task. Hence, investigation and mapping of the unseen characteristics of classification problems with the behaviors of classifiers is a key research problem focused in this research work. The characteristics of classification problems and behaviors of classifiers are measured in terms of meta-characteristics and classifiers performance, respectively. Machine learning research community has addressed the issue from time-to-time with various approaches, however the issues of – unavailability of training data, use of uni-metric evaluation criteria for classifiers evaluation, and selection of conflicting classifiers sill persist. Therefore, this research work has proposed a novel methodology based on edge ML and case-based reasoning (CBR) to overcome the aforementioned issues. The key contributions of the research work are enlisted as follows: (a) design of an incremental learning framework using edge ML-based CBR methodology, (b) design of a multi-metrics classifiers evaluation criteria, (c) design of an efficient algorithm conflict resolution criteria (ACR) and (d) implementation of the CBR methodology integrated with ACR to automatically select appropriate algorithm for new classification problems. To evaluate the proposed CBR-based algorithm recommendation method, jCollobri framework has been used and extensive experiments with 152 classification problems, in the form of classification datasets, and 09 decision tree classifiers are carried out. Results of the proposed multi-metrics performance criteria indicates that the proposed CBR-based methodology is an effective one as compared to the baseline classifier recommendation method and can be used in practical applications development classification and prediction problems.