Computer-assisted assignment of educational standards using natural language processing

Abstract

Educational standards are a central focus of the current educational system in the United States, underpinning educational practice, curriculum design, teacher professional development, and high-stakes testing and assessment. Digital library users have requested that this information be accessible in association with digital learning resources to support teaching and learning as well as accountability requirements. Providing this information is complex because of the variability and number of standards documents in use at the national, state, and local level. This article describes a cataloging tool that aids catalogers in the assignment of standards metadata to digital library resources, using natural language processing techniques. The research explores whether the standards suggestor service would suggest the same standards as a human, whether relevant standards are ranked appropriately in the result set, and whether the relevance of the suggested assignments improve when, in addition to resource content, metadata is included in the query to the cataloging tool. The article also discusses how this service might streamline the cataloging workflow. digital library services. The results of these focus groups indicate that teachers want standards information associated with classroom materials and prefer specific information such as detailed national standards or their particular state standards. Digital libraries can address this need by providing access to materials that have been associated with educational standards information and developing services that support the use of these materials in classroom settings. IntroductionOf the three main elements that make up a digital librarydocuments, technology, and work-, Levy & Marshall (1995) identify work as the most essential. Work is defined as the work of digital library users but also the work of librarians to support the users by making the library resources accessible. After all, a collection and supporting technology would be of no value if users could not find what they needed. The research presented here concerns both types of work: (a) the work of teachers to educate students according to certain state and national education standards, and (b) the work of catalogers to associate these standards with educational resources as metadata, thereby making the most relevant resources accessible.Associating educational standards with learning materials requires that two significant challenges be addressed.

Exploring educational standard alignment

“…Following these arguments, we submit that the poor inter-rater reliability reported in [2] and [1] could well be a methodological artifact; i.e., by not referring or appealing to the different and context-specific dimensions of 'relevance,' subjects partaking in the measurement are asked to formulate their own interpretation of these concepts. That many of them do so differently can be expected.…”

“…However, Devaul et al [2] present the results of an inter-rater reliability study of automated curriculum-standard alignment assessments. Thirty-five learning resources were independently aligned by two human experts with the help of the CAT system mentioned above.…”

“…Devaul et al [2], for instance, report that inter-rater reliability (the rate of agreement between different evaluators) for curriculum-standards alignments tends to be low, and hence, that these alignments might or might not be a good resource for standards-based searching. Similarly, Bar-Ilan et al [1] report on low inter-rater reliability of generic search engine results.…”

A Semi-Automated System for Recognizing Prior Knowledge

“…The objective of the tool was to remove redundancy in the repository. Authors in [20] presented a tool that automatically catalogues educational digital resources according to their contents and the educational standards of the educational system in their country.…”

The Relevance of K-12 Engineering Curricula to NGSS: An Analysis of TeachEngineering-NGSS Alignments (RTP Strand 1)

“…Inter-rater reliability problems are common in K-12 standards alignment exercises, as previously observed by, for instance, Devaul et al 8 , Reitsma et al 9 , and Reitsma and Diekema 6 .…”