Researchers are defining new types of interactions between humans and machine learning algorithms generically called human-in-the-loop machine learning. Depending on who is in control of the learning process, we can identify: active learning, in which the system remains in control; interactive machine learning, in which there is a closer interaction between users and learning systems; and machine teaching, where human domain experts have control over the learning process. Aside from control, humans can also be involved in the learning process in other ways. In curriculum learning human domain experts try to impose some structure on the examples presented to improve the learning; in explainable AI the focus is on the ability of the model to explain to humans why a given solution was chosen. This collaboration between AI models and humans should not be limited only to the learning process; if we go further, we can see other terms that arise such as Usable and Useful AI. In this paper we review the state of the art of the techniques involved in the new forms of relationship between humans and ML algorithms. Our contribution is not merely listing the different approaches, but to provide definitions clarifying confusing, varied and sometimes contradictory terms; to elucidate and determine the boundaries between the different methods; and to correlate all the techniques searching for the connections and influences between them.

No abstract

At present, the great majority of Artificial Intelligence (AI) systems require the participation of humans in their development, tuning, and maintenance. Particularly, Machine Learning (ML) systems could greatly benefit from their expertise or knowledge. Thus, there is an increasing interest around how humans interact with those systems to obtain the best performance for both the AI system and the humans involved. Several approaches have been studied and proposed in the literature that can be gathered under the umbrella term of Human-in-the-Loop Machine Learning. The application of those techniques to the health informatics environment could provide a great value on prognosis and diagnosis tasks contributing to develop a better health service for Cancer related diseases.

The use of Machine Learning (ML) techniques in the context of Cancer prognosis, di- agnosis and treatment is nowadays a reality. Some types of cancers could greatly benefit from specific techniques that are designed to work in a scarcity of data scenarios, or when obtaining labeled data is a time-consuming and/or costly task. It is the case of the Pan- creatic Adenocarcinoma. We present an experiment where Active Learning (AL) is used as the basis to create a model which performs a classification task where a human expert (in this experiment, a medical doctor) needs to determine whether a pancreatic cancer patient must be treated with chemotherapy, not treated, or he/she is unsure about the therapy. The use of AL techniques allows us to improve the accuracy of the model, and the inclusion of expert opinions may help us in the future to add explanatory capabilities to the system.

No abstract