Putting the Scientist in the Loop -- Accelerating Scientific Progress with Interactive Machine Learning

Aodha, Oisin Mac; Stathopoulos, Vassilios; Brostow, Gabriel J.; Terry, Michael; Girolami, Mark; Jones, Kate E.

doi:10.1109/icpr.2014.12

Cited by 20 publications

(17 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there are only few review papers that shape this area and review current work progress. A 15-step “data understanding pipeline” as the main steps of IML was outlined in Aodha et al [ 14 ] in 2014 for scientists to answer their research questions, where the questions strongly depend on data collection and data modelling. In this human–AI collaboration workflow, “hypothesis formation” is the first step and “publicising of results” is the last one.…”

Section: Introductionmentioning

confidence: 99%

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

Maadi

Khorshidi

Aickelin

2021

IJERPH

View full text Add to dashboard Cite

Objective: To provide a human–Artificial Intelligence (AI) interaction review for Machine Learning (ML) applications to inform how to best combine both human domain expertise and computational power of ML methods. The review focuses on the medical field, as the medical ML application literature highlights a special necessity of medical experts collaborating with ML approaches. Methods: A scoping literature review is performed on Scopus and Google Scholar using the terms “human in the loop”, “human in the loop machine learning”, and “interactive machine learning”. Peer-reviewed papers published from 2015 to 2020 are included in our review. Results: We design four questions to investigate and describe human–AI interaction in ML applications. These questions are “Why should humans be in the loop?”, “Where does human–AI interaction occur in the ML processes?”, “Who are the humans in the loop?”, and “How do humans interact with ML in Human-In-the-Loop ML (HILML)?”. To answer the first question, we describe three main reasons regarding the importance of human involvement in ML applications. To address the second question, human–AI interaction is investigated in three main algorithmic stages: 1. data producing and pre-processing; 2. ML modelling; and 3. ML evaluation and refinement. The importance of the expertise level of the humans in human–AI interaction is described to answer the third question. The number of human interactions in HILML is grouped into three categories to address the fourth question. We conclude the paper by offering a discussion on open opportunities for future research in HILML.

show abstract

Section: Introductionmentioning

confidence: 99%

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

Maadi

Khorshidi

Aickelin

2021

IJERPH

View full text Add to dashboard Cite

show abstract

“…Additionally, emerging low‐shot and zero‐shot visual learning approaches aim to learn classification models from very few examples of a class of interest, reducing the need for large training datasets (e.g., Hariharan & Girshick, ). More broadly, the limited understanding of the transferability of extant auto‐ID systems emphasises that, irrespective of the underlying algorithms, a critical focus must be on lowering the technical barriers to ecologists developing and testing bespoke tools, for example, via interactive machine learning software (Mac Aodha et al., ). Such functionality is beginning to emerge in bioacoustic analysis packages (e.g., Kaleidoscope, ARBIMON, Tadarida) (Aide et al., ; Bas et al., ).…”

Section: Detecting and Classifying Acoustic Signals Within Audio Datamentioning

confidence: 99%

Emerging opportunities and challenges for passive acoustics in ecological assessment and monitoring

et al. 2018

Self Cite

View full text Add to dashboard Cite

High‐throughput environmental sensing technologies are increasingly central to global monitoring of the ecological impacts of human activities. In particular, the recent boom in passive acoustic sensors has provided efficient, noninvasive, and taxonomically broad means to study wildlife populations and communities, and monitor their responses to environmental change. However, until recently, technological costs and constraints have largely confined research in passive acoustic monitoring (PAM) to a handful of taxonomic groups (e.g., bats, cetaceans, birds), often in relatively small‐scale, proof‐of‐concept studies. The arrival of low‐cost, open‐source sensors is now rapidly expanding access to PAM technologies, making it vital to evaluate where these tools can contribute to broader efforts in ecology and biodiversity research. Here, we synthesise and critically assess the current emerging opportunities and challenges for PAM for ecological assessment and monitoring of both species populations and communities. We show that terrestrial and marine PAM applications are advancing rapidly, facilitated by emerging sensor hardware, the application of machine learning innovations to automated wildlife call identification, and work towards developing acoustic biodiversity indicators. However, the broader scope of PAM research remains constrained by limited availability of reference sound libraries and open‐source audio processing tools, especially for the tropics, and lack of clarity around the accuracy, transferability and limitations of many analytical methods. In order to improve possibilities for PAM globally, we emphasise the need for collaborative work to develop standardised survey and analysis protocols, publicly archived sound libraries, multiyear audio datasets, and a more robust theoretical and analytical framework for monitoring vocalising animal communities.

show abstract

“…Further, they are now better positioned to accurately annotate the additional unlableled data outside the original teaching set. Automatic teaching algorithms have applications in many domains from education, language learning, medical image analysis, biological species identification [30], and more. Crucially, for automated teaching to be effective, it needs to be able to assess the student's current knowledge, and have a mechanism for selecting teaching examples to best improve this knowledge.…”

Section: Introductionmentioning

confidence: 99%

Becoming the expert - interactive multi-class machine teaching

Johns

Aodha

Brostow

2015

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

Compared to machines, humans are extremely good at classifying images into categories, especially when they possess prior knowledge of the categories at hand. If this prior information is not available, supervision in the form of teaching images is required. To learn categories more quickly, people should see important and representative images first, followed by less important images later -or not at all. However, image-importance is individual-specific, i.e. a teaching image is important to a student if it changes their overall ability to discriminate between classes. Further, students keep learning, so while image-importance depends on their current knowledge, it also varies with time.In this work we propose an Interactive Machine Teaching algorithm that enables a computer to teach challenging visual concepts to a human. Our adaptive algorithm chooses, online, which labeled images from a teaching set should be shown to the student as they learn. We show that a teaching strategy that probabilistically models the student's ability and progress, based on their correct and incorrect answers, produces better 'experts'. We present results using real human participants across several varied and challenging real-world datasets.

show abstract

Putting the Scientist in the Loop -- Accelerating Scientific Progress with Interactive Machine Learning

Cited by 20 publications

References 55 publications

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

Emerging opportunities and challenges for passive acoustics in ecological assessment and monitoring

Becoming the expert - interactive multi-class machine teaching

Contact Info

Product

Resources

About