Axis

Williams, Joseph Jay; Kim, Juho; Rafferty, Anna N.; Maldonado, Samuel; Gajos, Krzysztof Z.; Lasecki, Walter S.; Heffernan, Neil T.

doi:10.1145/2876034.2876042

Cited by 115 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Application-grounded evaluation is when the interpretability method is evaluated in the environment it will be deployed. For example, does the explanations result in higher survial-rates in a medical setting, or higher-grades in a homework-hint system (Doshi-Velez and Kim, 2017;Williams et al, 2016). Importantly, this evaluation should include the baseline where the explanations are provided by humans.…”

Section: Measures Of Interpretabilitymentioning

confidence: 99%

Post-hoc Interpretability for Neural NLP: A Survey

Madsen¹,

Reddy²,

Chandar³

2021

Preprint

View full text Add to dashboard Cite

Natural Language Processing (NLP) models have become increasingly more complex and widespread. With recent developments in neural networks, a growing concern is whether it is responsible to use these models. Concerns such as safety and ethics can be partially addressed by providing explanations. Furthermore, when models do fail, providing explanations is paramount for accountability purposes. To this end, interpretability serves to provide these explanations in terms that are understandable to humans. Central to what is understandable is how explanations are communicated. Therefore, this survey provides a categorization of how recent interpretability methods communicate explanations and discusses the methods in depth. Furthermore, the survey focuses on post-hoc methods, which provide explanations after a model is learned and generally model-agnostic. A common concern for this class of methods is whether they accurately reflect the model. Hence, how these post-hoc methods are evaluated is discussed throughout the paper.

show abstract

Section: Measures Of Interpretabilitymentioning

confidence: 99%

Post-hoc Interpretability for Neural NLP: A Survey

Madsen¹,

Reddy²,

Chandar³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The use of crowdsourcing in ALSs is also beginning to receive attention. For example, Heffernan et al (2016) propose employing crowdsourcing within the popular ASSISTments platform; Williams et al (2016) present an Adaptive eXplanation Improvement System (AXIS) that uses crowdsourcing to generate, revise, and evaluate explanations as learners solve problems; and Karataev and Zadorozhny (2017) propose a framework that combines concepts of crowdsourcing, online social networks, and adaptive systems to provide personalized learning pathways for students. However, this preliminary work is yet to realize the full potential offered by crowdsourcing in ALSs or more broadly in education.…”

Section: Content Repositorymentioning

confidence: 99%

RiPPLE: A Crowdsourced Adaptive Platform for Recommendation of Learning Activities

2019

Self Cite

View full text Add to dashboard Cite

This paper presents a platform called RiPPLE (Recommendation in Personalised Peer-Learning Environments) that recommends personalized learning activities to students based on their knowledge state from a pool of crowdsourced learning activities that are generated by educators and the students themselves. RiPPLE integrates insights from crowdsourcing, learning sciences, and adaptive learning, aiming to narrow the gap between these large bodies of research while providing a practical platform-based implementation that instructors can easily use in their courses. This paper provides a design overview of RiPPLE, which can be employed as a standalone tool or embedded into any learning management system (LMS) or online platform that supports the Learning Tools Interoperability (LTI) standard. The platform has been evaluated based on a pilot in an introductory course with 453 students at The University of Queensland. Initial results suggest that the use of the RiPPLE platform led to measurable learning gains and that students perceived the platform as beneficially supporting their learning. Notes for Practice• Adaptive learning systems (ALSs) dynamically adjust the level or type of instruction based on individual student abilities or preferences to provide a customized learning experience.• However, ALSs require access to a large repository of learning resources, which are commonly created by domain experts. This makes them expensive to develop and challenging to scale across different domains.• We introduce an ALS called RiPPLE that uses a crowdsourcing approach to engage students in creating, moderating, and evaluating learning resources. This both reduces the cost of content generation and has the potential to foster higher-order learning across many domains.• Initial results suggest that using RiPPLE may lead to measurable learning gains and that students perceived the platform as beneficially supporting their learning.

show abstract

“…In addition, novel techniques like crowdsourcing 18 enable applications in fields from social media to health to education where ideas are emerging after deployment. For example, Williams et al 19 show the values of having new explanations of a concept continually added by people learning a topic, once they can be tested out quickly to assess efficacy. More broadly, applications of experimentation for successful apps, websites and products rely on cycles of continual testing and improvement.…”

Section: Introductionmentioning

confidence: 99%

A flexible micro-randomized trial design and sample size considerations

Yan

Figueroa

et al. 2023

Stat Methods Med Res

Self Cite

View full text Add to dashboard Cite

Technological advancements have made it possible to deliver mobile health interventions to individuals. A novel framework that has emerged from such advancements is the just-in-time adaptive intervention, which aims to suggest the right support to the individuals when their needs arise. The micro-randomized trial design has been proposed recently to test the proximal effects of the components of these just-in-time adaptive interventions. However, the extant micro-randomized trial framework only considers components with a fixed number of categories added at the beginning of the study. We propose a more flexible micro-randomized trial design which allows addition of more categories to the components during the study. Note that the number and timing of the categories added during the study need to be fixed initially. The proposed design is motivated by collaboration on the Diabetes and Mental Health Adaptive Notification Tracking and Evaluation study, which learns to deliver effective text messages to encourage physical activity among patients with diabetes and depression. We developed a new test statistic and the corresponding sample size calculator for the flexible micro-randomized trial using an approach similar to the generalized estimating equation for longitudinal data. Simulation studies were conducted to evaluate the sample size calculators and an R shiny application for the calculators was developed.

show abstract

Axis

Cited by 115 publications

References 19 publications

Post-hoc Interpretability for Neural NLP: A Survey

Post-hoc Interpretability for Neural NLP: A Survey

RiPPLE: A Crowdsourced Adaptive Platform for Recommendation of Learning Activities

A flexible micro-randomized trial design and sample size considerations

Contact Info

Product

Resources

About