Automating Analysis and Feedback to Improve Mathematics Teachers’ Classroom Discourse

Suresh, A.; Sumner, Tamara; Jacobs, Jennifer; Foland, Bill; Ward, Wayne H.

doi:10.1609/aaai.v33i01.33019721

Cited by 22 publications

(21 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this context, the machine learning methods were designed to automatically assign codes from the behavioural coding measures to overt interactions recorded in the dataset (e.g., words/utteran- (Atkins et al, 2014;Can et al, 2015;Can et al, 2012;Can et al, 2016;Cao et al, 2020;Carcone et al, 2019, Study 1;Chakravarthula et al, 2015;Chen et al, 2019;Gibson et al, 2017;Gibson et al, 2016;Gupta et al, 2014;Hasan et al, 2019;Hasan et al, 2018;Imel et al, 2015;Perez-Rosas et al, 2017;Perez-Rosas et al, 2019;Singla et al, 2018;Tanana et al, 2016;Xiao et al, 2012;Xiao et al, 2015;Xiao, Can, et al, 2016;Xiao, Huang, et al, 2016) Medical care, provider-patient clinical interactions (Carcone et al, 2019, Study 2;Park et al, 2019) Education (teachers) (Blanchard et al, 2016a;Blanchard et al, 2016b;Donelly et al, 2017;Donnely et al, 2016a;Donnelly et al, 2016b;Samei et al, 2014;Samei et al, 2015;Song et al, 2020;Suresh et al, 2019;Wang et al, 2014) Counselling, (counsellors), (Althoff et al, 2016;Flemotomos et al, 2018;Gallo et al, 2015;Gaut et al, 2017;Goldberg et al, 2020…”

Section: Synthesis Of Resultsmentioning

confidence: 99%

“…Hidden Markov Model 4 Althoff et al, 2016;Can et al, 2012;Hasan et al, 2019;Hasan et al, 2018K-Nearest Neighbours 4 Blanchard et al, 2016aSen et al, 2017;Blanchard et al, 2016b;Donnelly et al, 2017 Conditional Random Field 4 Can et al, 2015;Carcone et al, 2019;Wallace et al, 2014;Park et al, 2019 Bi-directional Long Short Term Memory (Bi-LSTM) 3 Chen et al, 2019;Suresh et al, 2019 Labelled Topic Model 2 Atkins et al, 2014;Imel et al, 2015 Bayesian Network 2 Blanchard et al, 2016a;Blanchard et al, 2016b Gared Recurrent Unit (GRU) 2 Cao et al, 2019;Park et al, 2019 30 models were used once each 2 1 each (30 in total) Angus et al, 2012;Carcone et al, 2019;Chakravarthula et al, 2015;Gallo et al, 2015;Gaut et al, 2017;Gibson et al, 2016;Gibson et al, 2017;Hasan et al, 2018;Howes et al, 2013;Imel et al, 2015;Lacson et al, 2005;Malandrakis and Narayanan, 2015;Nitti et al, 2010;Salvatore et al, 2012;Tanana et al, 2016;Velasquez & Montiel, 2018;Wallace et al, 2013;Xiao et al, 2012;Xiao, Huang et al, 2016;Xiao, Can, et al, 2016;…”

Section: Article In Pressmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

Ahmadi¹,

Noetel²,

Schellekens³

et al. 2021

Preprint

View full text Add to dashboard Cite

Many psychological treatments have been shown to be cost-effective and efficacious, as long as they are implemented faithfully. Assessing fidelity and providing feedback is expensive and time-consuming. Machine learning has been used to assess treatment fidelity, but the reliability and generalisability is unclear. We collated and critiqued all implementations of machine learning to assess the verbal behaviour of all helping professionals, with particular emphasis on treatment fidelity for therapists. We conducted searches using nine electronic databases for automated approaches of coding verbal behaviour in therapy and similar contexts. We completed screening, extraction, and quality assessment in duplicate. Fifty-two studies met our inclusion criteria (65.3% in psychotherapy). Automated coding methods performed better than chance, and some methods showed near human-level performance; performance tended to be better with larger data sets, a smaller number of codes, conceptually simple codes, and when predicting session-level ratings than utterance-level ones. Few studies adhered to best-practice machine learning guidelines. Machine learning demonstrated promising results, particularly where there are large, annotated datasets and a modest number of concrete features to code. These methods are novel, cost-effective, scalable ways of assessing fidelity and providing therapists with individualised, prompt, and objective feedback.

show abstract

Section: Synthesis Of Resultsmentioning

confidence: 99%

Section: Article In Pressmentioning

confidence: 99%

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

Ahmadi¹,

Noetel²,

Schellekens³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Our work is a first step towards improving in-class discussions with the help of an NLP system and, thus, to improve student learning and engagement. Prior work in understanding classroom discourse using NLP includes Suresh et al (2019) and Donnelly et al (2016). They propose an application where feedback can be provided to teachers by automatically classifying their utterances into talk moves.…”

Section: Nlp For Educational Applicationsmentioning

confidence: 99%

“…For our experiments, we make use of the dataset from Suresh et al (2019). It consists of 216 annotated transcripts of classroom discourse collected in public schools in the US.…”

Section: Datasetmentioning

confidence: 99%

What Would a Teacher Do? Predicting Future Talk Moves

Ganesh¹,

Palmer²,

Kann³

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Recent advances in natural language processing (NLP) have the ability to transform how classroom learning takes place. Combined with the increasing integration of technology in today's classrooms, NLP systems leveraging question answering and dialog processing techniques can serve as private tutors or participants in classroom discussions to increase student engagement and learning. To progress towards this goal, we use the classroom discourse framework of academically productive talk (APT) to learn strategies that make for the best learning experience. In this paper, we introduce a new task, called future talk move prediction (FTMP): it consists of predicting the next talk move -an utterance strategy from APT -given a conversation history with its corresponding talk moves. We further introduce a neural network model for this task, which outperforms multiple baselines by a large margin. Finally, we compare our model's performance on FTMP to human performance and show several similarities between the two.

show abstract

“…The capabilities, however, are going beyond observing what students type on a computer or how they answer questions. Newer research-based systems can listen to recordings or watch videos of classrooms, finding events that are significant for learning outcomes (Suresh et al, 2019;Aung et al, 2018). Automated essay scoring is another long-standing application (Page, 2003), which is now rapidly expanding to include assistive systems for peer grading, student collaboration, and other educational applications.…”

Section: Defining Artificial Intelligencementioning

confidence: 99%

AI and the Future of Learning: Expert Panel Report

Roschelle¹,

Lester²,

Fusco³

2020

View full text Add to dashboard Cite

This report is based on the discussion that emerged from a convening of a panel of 22 experts in artificial intelligence (AI) and in learning. It introduces three layers that can frame the meaning of AI for educators. First, AI can be seen as “computational intelligence” and capability can be brought to bear on educational challenges as an additional resource to an educator’s abilities and strengths. Second, AI brings specific, exciting new capabilities to computing, including sensing, recognizing patterns, representing knowledge, making and acting on plans, and supporting naturalistic interactions with people. Third, AI can be used as a toolkit to enable us to imagine, study, and discuss futures for learning that don’t exist today. Experts voiced the opinion that the most impactful uses of AI in education have not yet been invented. The report enumerates important strengths and weaknesses of AI, as well as the respective opportunities and barriers to applying AI to learning. Through discussions among experts about these layers, we observed new design concepts for using AI in learning. The panel also made seven recommendations for future research priorities.

show abstract

Automating Analysis and Feedback to Improve Mathematics Teachers’ Classroom Discourse

Cited by 22 publications

References 20 publications

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

What Would a Teacher Do? Predicting Future Talk Moves

AI and the Future of Learning: Expert Panel Report

Contact Info

Product

Resources

About