A top-level model of case-based argumentation for explanation: Formalisation and experiments

Abstract

This paper proposes a formal top-level model of explaining the outputs of machine-learning-based decision-making applications and evaluates it experimentally with three data sets. The model draws on AI & law research on argumentation with cases, which models how lawyers draw analogies to past cases and discuss their relevant similarities and differences in terms of relevant factors and dimensions in the problem domain. A case-based approach is natural since the input data of machine-learning applications can be seen as cases. While the approach is motivated by legal decision making, it also applies to other kinds of decision making, such as commercial decisions about loan applications or employee hiring, as long as the outcome is binary and the input conforms to this paper’s factor- or dimension format. The model is top-level in that it can be extended with more refined accounts of similarities and differences between cases. It is shown to overcome several limitations of similar argumentation-based explanation models, which only have binary features and do not represent the tendency of features towards particular outcomes. The results of the experimental evaluation studies indicate that the model may be feasible in practice, but that further development and experimentation is needed to confirm its usefulness as an explanation model. Main challenges here are selecting from a large number of possible explanations, reducing the number of features in the explanations and adding more meaningful information to them. It also remains to be investigated how suitable our approach is for explaining non-linear models.

Using Issues to Explain Legal Decisions

“…Note, however, that it may be that neither F6p or F19d is present in the case: even if security measures were taken, so that there is no reason to find for the defendant on this aspect, they may not have been sufficient to provide a reason to find for the plaintiff, and so that the aspect is neutral. The factors from [3] have been reused by many subsequent researchers, including [19], [21], [1], [55] and [42].…”

“…This formalisation uses the results model, and uses the full set of factors available to both sides. These schemes were proposed as a means of providing explanation for ML systems in [42]. We will discuss the explanations from [42] in the next section.…”

“…These schemes were proposed as a means of providing explanation for ML systems in [42]. We will discuss the explanations from [42] in the next section.…”

“…Explanation in [42] takes the form of a dialogue modelling the three ply argumentation structure of CATO. An example dialogue is shown in Figure 1.…”

“…In [17] the idea is to first ascribe factors to cases using ML and then to explain the outcomes in terms of these factors. In [42], the proposal is to produce the explanation from a case based system running in parallel with the ML system.…”