Detecting learning in noisy data

Klebanov, Beata Beigman; Loukina, Anastassia; Lockwood, J. R.; Liceralde, Van Rynald T.; Sabatini, John; Madnani, Nitin; Gyawali, Binod; Wang, Zuowei; Lentini, Jennifer

doi:10.1145/3375462.3375490

Cited by 3 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results suggest that the acoustic features used in this study can predict the human-rated ORF for upper elementary students well. The models using support vector machine or random forest algorithms showed a higher correlation coefficient between the human-machine ORF estimates (r =.91) than those documented in the previous studies that involved ASR-dependent ORF assessment: .78 for Grades 1-4 in Beck, Peng and Mostow (2004) and .75 for Grades 4-5 in Klebanov et al (2020). More importantly, we found no evidence that supports different model performance for so-to-speak accented speech.…”

Section: Discussioncontrasting

confidence: 86%

Survey datasets of VIKKE assessment tool on children's Finnish Sign Language development

Syrj�l�,

Kanto

2024

Jyx

View full text Add to dashboard Cite

Taking as a case in point Cypriot Arabic (CyAr), a severely endangered language spoken in Cyprus, this paper discusses the results of the research project MapCyArS (Mapping Cypriot Arabic speakers: An investigation into linguistic demography and the sociolinguistic profile of Kormakiote Maronites), funded by the A G Leventis Foundation at the University of Cyprus (2017-2020), and highlights the need to change the way we consider language testing and assessment to fit the sociolinguistic situation of less widely spoken, oral and/or endangered languages and their communities. In the case of CyAr, we need to consider the fact that it is a language of oral tradition that has suffered severe reduction on all linguistic levels and it is not a first language for people under 40 anymore.1 Although the CEFR is designed also for lesser used and taught languages, it has nevertheless been applied to very few, among them: Catalan, Galician, Basque, Friulan or Welsh, etc., which are officially recognized within their respective areas of use. Conversation Verbal L2 vs Verbal L1The task consists of using CyGr or translanguaging to provide short answers to questions asked in CyAr LEVEL 1 LEVEL 0 Fails to understand CyAr and correctly respond in SG 3 /(Cy)Gr Correctly understands CyAr and responds in SG/(Cy)Gr 5 Conversation Verbal L2 vs Verbal L2 The task consists of using CyAr to provide short answers to questions asked in CyAr LEVEL 1 LEVEL 2 LEVEL 0 Fails to understand and correctly respond in CyAr Correctly understands common sentences and responds in CyAr Correctly understands a wide range of conversation 6 Metalinguistic/ Grammatical Verbal L2 vs Verbal L2The task consists of completing orally metalinguistic or grammatical tasks such as declining a verb, giving the plural of a noun, translating etc. LEVEL 1 LEVEL 2 LEVEL 0Unable to accomplish any metalinguistic task Able to accomplish simple metalinguistic tasks Able to accomplish complex metalinguistic tasks Conclusion 3Designing an assessment test for an endangered oral language like CyAr is a complex objective that needs to take into consideration the oral character of CyAr, its endangered situation and the advanced state of its replacement by (Cy)Gr.The oral character of CyAr suggests that we move away from reading and writing as means to evaluate oral skills. Its endangered situation suggests that traditional approaches in testing and assessment by level of competence (A1, A2, etc.) should be abandoned in favour of a more flexible type of evaluation. The advanced state of CyAr replacement by (Cy)Gr leads towards a more flexible use of language resources to evaluate speakers' proficiency, and invites us to capitalize on the use of (Cy)Gr or translanguaging to do so. The project is, therefore, also an opportunity to re-examine the baselines of language assessment used for modern languages in order to adapt them to conditions of orality and severe endangerment.

show abstract

Section: Discussioncontrasting

confidence: 86%

Survey datasets of VIKKE assessment tool on children's Finnish Sign Language development

Syrj�l�,

Kanto

2024

Jyx

View full text Add to dashboard Cite

show abstract

“…In the current study, we use a liberal definition of "learning," which included any measure of grades, scores, performance, or completion. Exam scores were the most common form, but our definition also included other measures such as reading fluency (Klebanov et al, 2020) and the depth of student reflections as rated on an established rubric (Carpenter et al, 2021); thus we emphasize that studies using noncognitive learning outcomes (Joksimović et al, 2020) were coded as measuring learning, to the extent that these outcomes were directly and deliberately assessed. These, and any other measures of learning, involve a set of assumptions about knowledge and meaning (Knight et al, 2014); our approach is intended to be pluralistic across such assumptions.…”

Section: Discussionmentioning

confidence: 99%

LAK of Direction

et al. 2023

View full text Add to dashboard Cite

Learning analytics defines itself with a focus on data from learners and learning environments, with corresponding goals of understanding and optimizing student learning. In this regard, learning analytics research, ideally, should be characterized by studies that make use of data from learners engaged in education systems, should measure student learning, and should make efforts to intervene and improve these learning environments. However, a common concern among members of the learning analytics research community is that these standards are not being met. In two analysis waves, we review a large and comprehensive sample of research articles from the proceedings of the three most recent Learning Analytics and Knowledge conferences, the premier conference venue for learning analytics research, and from articles published during the same time in the Journal of Learning Analytics (over the years of 2020, 2021, and 2022). We find that 37.4% of articles do not analyze data from learners in an education system, 71.1% do not include any measure of learning, and 89.0% of articles do not attempt to intervene in the learning environment. We contrast these findings with the stated definition of learning analytics and infer, like others before us, that scholarship in learning analytics research presently lacks clear direction toward its stated goals. We invite critical discussion of these findings from the learning analytics community, through open peer commentary.

show abstract

“…The PlushPal project utilizes gesturing robotic toys to encourage programmatic thinking (C1.3) in this population of students (Tseng et al, 2021). Relay Reader is an oral reading fluency tool in which young readers alternate between listening and reading aloud throughout the course of a book (A1.3, A2.3) (Klebanov et al, 2020). The ITSS is an intelligent tutoring system for late elementary education in which focuses on teaching discourse elements like genre and text structures (B3) (Meyer & Wijekumar, 2007).…”

Section: Brief Description Of Exemplary Ai Research and Products Alig...mentioning

confidence: 99%

A framework of literacy development and how AI can transform theory and practice

Sabatini

Graesser

Hollander

et al. 2023

Brit J Educational Tech

Self Cite

View full text Add to dashboard Cite

We argue in this paper that there is currently no adequate theoretical framework or model that spans the twelve odd year trajectory from non‐reader to proficient reader, nor addresses fine‐grain skill acquisition, mastery and integration. The target construct itself, reading proficiency, as often operationalized as an endpoint of formal secondary schooling, is defined and measured in imprecise, fragmented terms. Consequently, schools (and empirical research) fall back on heuristics like the Simple View of Reading, or a few stages (learn to read, read to learn, read to do) to describe reading development. Those models, however, are too general to guide instructional decisions for adaptive learning systems. Progress in engineering an adequate learning system has been inhibited by a mismatch with curriculum standards and school organization that impose good‐faith but not fully optimized developmental targets on the educational system. We propose the development of a learning and assessment framework to scaffold reading proficiency development while accounting for the diverse learning trajectories of groups or individuals across development. We then identify some key problems, challenges and opportunities that AI technologies are poised to help us address in conceiving individualized, adaptive learning systems for reading proficiency across the developmental spectrum. We close with a selective review of examples of AI‐enhanced research or products. Practitioner notesWhat is already known about this topic Reading proficiency is a vital component of education systems design. Theoretical and empirical studies across multiple disciplines have been published, but much of this research is framed in fragmented theories that do not seamlessly span the trajectory of life‐long reading development. Current assessments are time‐consuming, coarse‐grained and fail to provide a roadmap for educators and designers of adaptive learning environments. There is a significant lack of knowledge of potentially non‐linear growth in reading skills within or between years and how to design adaptive instruction for diverse subpopulations. What this paper adds We describe the initial steps towards a literacy learning and assessment framework that spans the trajectory from non‐reader to proficient reader. We provide a landscape of exemplars of artificial intelligence and computational linguistics that reflect the possibilities of a more comprehensive, cohesive literacy development system. We reflect upon key problems, challenges and opportunities that AI technologies can help address in conceiving individualized, adaptive learning systems for reading proficiency across the developmental spectrum. Implications for practice and/or policy AI and computational linguistics can help fill in the gaps in understanding and enacting a longitudinal vision of reading development. Educators would ideally know what to expect of their students at particular points of development, identify deviations and have additional tools to intervene effectively to maximize progress. There is a need to develop adaptive instruction that spans the development of proficiency from preschool to college/career levels and adapts to address common barriers among diverse subpopulations.

show abstract

Detecting learning in noisy data

Cited by 3 publications

References 12 publications

Survey datasets of VIKKE assessment tool on children's Finnish Sign Language development

Survey datasets of VIKKE assessment tool on children's Finnish Sign Language development

LAK of Direction

A framework of literacy development and how AI can transform theory and practice

Contact Info

Product

Resources

About