Evaluating CBR Systems Using Different Data Sources: A Case Study

Abstract

Abstract. The complexity and high construction cost of case bases make it very difficult, if not impossible, to evaluate a CBR system, especially a knowledge-intensive CBR system, using statistical evaluation methods on many case bases. In this paper, we propose an evaluation strategy, which uses both many simple case bases and a few complex case bases to evaluate a CBR system, and show how this strategy may satisfy different evaluation goals. The identified evaluation goals are classified into two categories: domain-independent and domain-dependent. For the evaluation goals in the first category, we apply the statistical evaluation method using many simple case bases (for example, UCI data sets); for evaluation goals in the second category, we apply different, relatively weak, evaluation methods on a few complex domain-specific case bases. We apply this combined evaluation strategy to evaluate our knowledge-intensive conversational CBR method as a case study.

Cognitive Management of Bandwidth Allocation Models with Case-Based Reasoning - Evidences Towards Dynamic BAM Reconfiguration

“…As discussed in [27], cognitive management research is an empirical process in which a management task is selected to incorporate intelligence features, a management system is built incorporating these features and the system is evaluated with different management task scenarios. Cognitive systems evaluation methods include statistical evaluation, theoretical analysis, tuning evaluation, limitation evaluation and characteristic analysis, among other alternatives [28].…”

“…In the perspective of BAMCBR evaluation, [27] also points out that typical complexity of CBR application contexts makes CBR solution evaluation difficult, if not impossible.…”

Case-Based Reasoning for Situation-Aware Ambient Intelligence: A Hospital Ward Evaluation Study

“…This is a well-established approach [15], and also the assessment strategy adopted to assess the AmICreek system. As pointed out in [24], however, evaluating the quality of case-based reasoning systems may not be as straightforward, for example when cases are complex structures which are time consuming to construct, or when the user benefits from a larger part of a rich case than just a particular value. These concerns should also make some bells ring when building and evaluating user-interactive systems in health care.…”