Developing and Applying an Information Literacy Rubric to Student Annotated Bibliographies

Abstract

Objective – This study demonstrates one method of developing and applying rubrics to student writing in order to gather evidence of how students utilize information literacy skills in the context of an authentic assessment activity. The process of creating a rubric, training scorers to use the rubric, collecting annotated bibliographies, applying the rubric to student work, and the results of the rubric assessment are described. Implications for information literacy instruction are also discussed.
 
 Methods – The focus of this study was the English 102 (ENG 102) course, a required research-based writing course that partners the instructors with the university librarians for information literacy instruction. The author developed an information literacy rubric to assess student evaluation of information resources in the ENG 102 annotated bibliography assignment and trained three other librarians how to apply the rubric to student work. The rubric assessed the extent to which students critically applied the evaluative criteria Currency, Relevance, Accuracy, Authority, and Purpose to the information sources in their annotations. At the end of the semester, the author collected up to three de-identified annotated bibliographies from each of the 58 sections of ENG 102. The rubric was applied to up to five annotations in each bibliography, resulting in a total examination of 773 annotations (some sections turned in fewer than 3 samples, and some bibliographies had fewer than 5 annotations).
 Results – The results of the study showed that students struggle with critically evaluating information resources, a finding that supports the existing information literacy assessment literature. The overwhelming majority of annotations consisted of summative information with little evidence that students used any evaluative criteria when they selected an information source. Of the five criteria examined, Relevance to the student’s research topic and Authority were the most commonly used methods of resource evaluation, while Currency, Accuracy, and Purpose were criteria least-often used. The low average scores on the rubric assessment indicate that students are not adequately learning how to apply this set of information literacy skills.
 
 Conclusions – The library instruction sessions for ENG 102 need to move beyond the skills of choosing and narrowing a topic, selecting keywords, and searching in a library database. Students also need more targeted instruction on higher-order skills, particularly how to critically evaluate and question the sources they find. The results of this assessment are being used to refocus the learning outcomes of ENG 102 library sessions so that instruction can better meet student needs. The results are also being used to make the case for further collaboration between ENG 102 and the university library.

Applying an information literacy rubric to first-year health sciences student research posters

“…The assignment required students to: (1) locate peer-reviewed research articles; (2) evaluate the sources for relevance to their topics; (3) summarize the sources, including their research methods and conclusion; and (4) cite the sources using APA style. To design the rubric, we reviewed guidelines for information literacy rubrics [ 6 , 8 ] and settled on four performance levels (1–4) representing a range of student abilities to meet each of the four rubric criteria ( Table 1 ).…”

Library Instruction and Themed Composition Courses: An Investigation of Factors that Impact Student Learning

“…The author used a training process based on the recommendations of Maki (2010) and Oakleaf (2009). The author introduced the annotated bibliography assignment to the group, explained the rubric and criteria, and conducted a read-aloud of several sample papers so the group could see how to apply the rubric (Rinto, 2013). Each librarian then individually scored an identical set of student work, the group discussed differences in scoring, and worked toward a shared understanding of how to apply the rubric.…”

“…The literature on authentic assessment provides a great deal of information on best practices for creating and applying rubrics (Arter & McTighe, 2001;Maki, 2010;Mertler, 2001;Moskal, 2003;Stevens & Levi, 2005), including studies specifically on the development of rubrics in an information literacy context (Fagerheim & Shrode, 2009;Hoffman & LaBonte, 2012;Oakleaf, 2009). The two rubrics were created using these best practices, including selection of outcomes, articulation of what characterizes student work as exemplary, average, or poor for each outcome, and revising the rubrics based on test applications of student work (Rinto, 2013).…”

“…As with the Proposal rubric, there were three performance indicators to represent the range of student work: "level 0-not evidenced," "level 1-developing (using evaluation criteria at face value)," and "level 2-competent (using evaluation criteria critically)". The goal of this rubric was to identify to what extent, if at all, students evaluated their information sources (Rinto, 2013). The Annotated Bibliography rubric was designed to evaluate the individual annotations in each bibliography, and was applied to each student's work five times, since students were to complete at least five annotations.…”

Undergraduates’ assessment of Science, Technology, Engineering and Mathematics (STEM) information literacy instruction

“…Carbery and Leahy (2015), Mueller (2014), Booth et al (2014) and Biggs and Tang (2011) [order] highlight authentic assessment as a viable ILI assessment option. Carbery and Leahy (2015) and Rinto (2013) cite rubrics and annotated bibliographies as examples of authentic assessment. Mueller (2014) also cites rubrics and adds scoring scales and portfolios as further examples; Bennett and Brothen (2010) note citation analyses as an option as well.…”