The Use of a Prototype System for Evaluating Knowledge Elicitation Techniques

Abstract

This paper describes the final phase of a research project aimed at investigating the efficacy of using two distinct methods of knowledge elicitation, comparing the knowledge obtained via classification or scaling techniques with that derived from the more traditional interview based approaches. It outlines the process of evaluating the knowledge elicited from an expert via a prototype specifically created for this purpose, the design of which was based directly on the findings of previous rapid prototyping approaches. The prototype was modelled to facilitate the evaluation of elicited knowledge along a numerical scale according to predetermined criteria. The use of the prototype resulted in the knowledge elicited via scaling methods receiving higher ratings than that obtained via non-scaling methods on the criteria of 'correctness' and 'relevance', although no significant difference was found between the two sets of data on the criterion of 'suitable for a knowledge based system'.

Formalizing a Systematic Review Updating Process

“…To collect this information we can apply different requirements elicitation techniques [27]. However, despite the existence of a large set of techniques, there are not clear guidelines which define when to apply those techniques in practice [23,43].…”

Automatic Formulation of the Internal Control Report over Financial Reporting

“…Knowledge acquiring is the extraction of knowledge from experts. It is a primary bottleneck to the development of dependable and practically usable KBSs (Jones and Miles, 1998). Therefore, the knowledge needed to construct ARICOFR was attained from the literature on ISA (IFAC, 2010), academic resources, journals, and from the qualified auditors.…”

Improving the use of Visual Interactive Simulation as a knowledge elicitation tool

“…Under Hypothesis 1, the view is that increased visual fidelity in VIS-based knowledge elicitation will improve the KBS' fidelity to real-life decision-making (primary effectiveness). This hypothesis can be tested using a single blind evaluation test (Jones and Miles 1998), where a selection of decision cases collected earlier from every decision-maker under each factor-level combination will be presented back to them in a randomised order. Every decision-maker will then be asked to rate how strongly he agrees with each decision case.…”