Computer-Assisted Reading and Discovery for Student Generated Text in Massive Open Online Courses

Abstract

ABSTRACT:Dealing with the vast quantities of text that students generate in Massive Open Online Courses (MOOCs) and other large-scale online learning environments is a daunting challenge. Computational tools are needed to help instructional teams uncover themes and patterns as students write in forums, assignments, and surveys. This paper introduces to the learning analytics community the Structural Topic Model, an approach to language processing that can 1) find syntactic patterns with semantic meaning in unstructured text, 2) identify variation in those patterns across covariates, and 3) uncover archetypal texts that exemplify the documents within a topical pattern. We show examples of computationally aided discovery and reading in three MOOC settings: mapping students' self-reported motivations, identifying themes in discussion forums, and uncovering patterns of feedback in course evaluations.

stm: An R Package for Structural Topic Models

“…Building off of the tradition of probabilistic topic models, such as the latent Dirichlet allocation model (LDA; Blei, Ng, and Jordan 2003), the correlated topic model (CTM; Blei and Lafferty 2007), and other topic models that have extended these (Mimno and McCallum 2008;Socher, Gershman, Perotte, Sederberg, Blei, and Norman 2009;Eisenstein, O'Connor, Smith, and Xing 2010;Rosen-Zvi, Chemudugunta, Griffiths, Smyth, and Steyvers 2010;Quinn, Monroe, Colaresi, Crespin, and Radev 2010;Ahmed and Xing 2010;Grimmer 2010;Eisenstein, Ahmed, and Xing 2011;Gerrish and Blei 2012;Foulds, Kumar, and Getoor 2015;Paul and Dredze 2015), the structural topic model's key innovation is that it permits users to incorporate arbitrary metadata, defined as infor-mation about each document, into the topic model. With the STM, users can model the framing of international newspapers (Roberts, Stewart, and Airoldi 2016b), open-ended survey responses in the American National Election Study (Roberts et al 2014), online class forums (Reich, Tingley, Leder-Luis, Roberts, and Stewart 2015), Twitter feeds and religious statements (Lucas, Nielsen, Roberts, Stewart, Storer, and Tingley 2015), lobbying reports (Milner and Tingley 2015) and much more. 1 The goal of the structural topic model is to allow researchers to discover topics and estimate their relationship to document metadata.…”

The Civic Mission of MOOCs: Engagement across Political Differences in Online Forums

“…Thus, it is possible our ngram-based analysis could miss the effect of partisanship broader trends in topic use. To examine this we use a form of unsupervised text analysis from the topic modeling tradition (Blei, Ng & Jordan, 2003) called the Structural Topic Model (Roberts et al, 2014; Reich et al, 2015; Roberts, Stewart & Airoldi, 2016). Topic models are designed to identify sets of words, “topics,” that tend to occur together.…”

Knowledge Discovery from the Programme for International Student Assessment

“…Slightly different variants were later offered to characterize the discipline (Pardo & Teasley 2014, Gray et al 2014, Siemens & Baker 2012. Increased attention to Massive Open Online Courses (MOOCs) (e.g., Wang et al 2014, Ye & Biswas 2014, Reich et al 2014, Coffrin et al 2014, Santos et al 2014, Vogelsang and Ruppertz 2015, Ferguson and Clow 2015, Hansen and Reich 2015, Wise et al 2016, Hecking et al 2016) has intensified the need for data-based learning support from the perspective of big data. This is evidenced by several articles (e.g., Picciano 2012, Chatti et al 2012, Siemens 2012, Chatti et al 2014, Wise & Shaffer 2015, Merceron et al 2016 as well as by the theme of the 2015 Learning Analytics and Knowledge conference "Scaling Up: Big Data to Big Impact" (see Dawson et al 2015).…”