Automatically Extracting Topical Components for a Response-to-Text
            Writing Assessment

Rahimi, Zahra; Litman, Diane

doi:10.18653/v1/w16-0532

Cited by 6 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…T C attn is our proposed method and automatically extracts TCs using both a source text and student essays. T C lda (Rahimi and Litman, 2016) (baseline) builds on LDA to extract TCs from student essays only, while T C pr (baseline) builds on PositionRank (Florescu and Caragea, 2017) traction, we needed to further process its output to create T C pr . To extract topic words, we extract all keywords from the output.…”

Section: Experimental Setup and Resultsmentioning

confidence: 99%

“…Methods for extracting important keywords or keyphrases also exist, both supervised (unlike our approach) (Meng et al, 2017;Mahata et al, 2018;Florescu and Jin, 2018) and unsupervised (Florescu and Caragea, 2017). Rahimi and Litman (2016) developed a TC extraction LDA model (Blei et al, 2003). While the LDA model considers all words equally, our model takes essay scores into account by using attention to represent word importance.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Automated Topical Component Extraction Using Neural Network Attention Scores from Source-based Essay Scoring

Zhang

Litman

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

While automated essay scoring (AES) can reliably grade essays at scale, automated writing evaluation (AWE) additionally provides formative feedback to guide essay revision. However, a neural AES typically does not provide useful feature representations for supporting AWE. This paper presents a method for linking AWE and neural AES, by extracting Topical Components (TCs) representing evidence from a source text using the intermediate output of attention layers. We evaluate performance using a feature-based AES requiring TCs. Results show that performance is comparable whether using automatically or manually constructed TCs for 1) representing essays as rubric-based features, 2) grading essays.

show abstract

Section: Experimental Setup and Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Automated Topical Component Extraction Using Neural Network Attention Scores from Source-based Essay Scoring

Zhang

Litman

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…While we have developed pilot data-driven methods that can extract such topical components automatically (Rahimi and Litman 2016), our methods need to be improved so that they do not degrade SG model performance. eRevise will also be enhanced to provide feedback for Organization, a second substantive RTA writing dimension for which we already have a pilot AES (Rahimi et al 2017).…”

Section: Current and Future Directionsmentioning

confidence: 99%

eRevise: Using Natural Language Processing to Provide Formative Feedback on Text Evidence Usage in Student Writing

Zhang

Magooda

Litman

et al. 2019

AAAI

View full text Add to dashboard Cite

Writing a good essay typically involves students revising an initial paper draft after receiving feedback. We present eRevise, a web-based writing and revising environment that uses natural language processing features generated for rubricbased essay scoring to trigger formative feedback messages regarding students' use of evidence in response-to-text writing. By helping students understand the criteria for using text evidence during writing, eRevise empowers students to better revise their paper drafts. In a pilot deployment of eRevise in 7 classrooms spanning grades 5 and 6, the quality of text evidence usage in writing improved after students received formative feedback then engaged in paper revision.

show abstract

“…Southavilay, Yacef, Reimann, and Calvo (2013) used topic modeling to create topic evolution charts to better understand how the topics in collaborative student writing changed over time when students used Google Docs to write together. Kakkonen, Myller, and Sutinen (2006) and Rahimi and Litman (2016) describe employing LDA to assist with automatic scoring of essays. Because of the high enrollment of Massive Open Online Courses (MOOCs), topic modeling is useful for quickly analyzing course forums, a task that would be nearly impossible to do manually.…”

Section: Topic Modelingmentioning

confidence: 99%

First-Year Students and the Framework: Using Topic Modeling to Analyze Student Understanding of the Framework for Information Literacy for Higher Education

Harden

2019

EBLIP

View full text Add to dashboard Cite

Abstract Objective – The Framework for Information Literacy for Higher Education has generated a significant amount of discussion among academic librarians; however, few have discussed the potential impact on learning when students interact directly with the Framework itself. At the University of Notre Dame, over 1,900 first-year students completed an information literacy assignment in their required first-year experience course. Students read a condensed version of the Framework, then wrote a response discussing how a frame of their choosing was reflected in an assigned reading. The goal of this exploratory study was to determine if the students demonstrated an understanding of the themes and concepts in the Framework based on this assignment. Methods – Topic modeling, a method for discovering topics contained in a corpus of text, was used to explore the themes that emerged in the students’ responses to this assignment and assess the degree to which they connect to frames in the Framework. The model receives no information about the Framework prior to the analysis; it only uses the students’ words to form topics. Results – The responses formed several topics that are recognizable as related to the frames from the Framework, suggesting that students were able to engage effectively and meaningfully with the language of the Framework. Because the topic model does not know anything about the Framework, the fact that the responses formed topics that are recognizable as frames suggests that students internalized the concepts in the Framework well enough to express them in their own writing. Conclusion – This research provides insight regarding the impact that the Framework may have on student understanding of information literacy concepts.

show abstract

Automatically Extracting Topical Components for a Response-to-Text Writing Assessment

Cited by 6 publications

References 14 publications

Automated Topical Component Extraction Using Neural Network Attention Scores from Source-based Essay Scoring

Automated Topical Component Extraction Using Neural Network Attention Scores from Source-based Essay Scoring

eRevise: Using Natural Language Processing to Provide Formative Feedback on Text Evidence Usage in Student Writing

First-Year Students and the Framework: Using Topic Modeling to Analyze Student Understanding of the Framework for Information Literacy for Higher Education

Contact Info

Product

Resources

About