Norming a VALUE rubric to assess graduate information literacy skills

Abstract

Objective: The study evaluated whether a modified version of the information literacy Valid Assessment of Learning in Undergraduate Education (VALUE) rubric would be useful for assessing the information literacy skills of graduate health sciences students.Methods: Through facilitated calibration workshops, an interdepartmental six-person team of librarians and faculty engaged in guided discussion about the meaning of the rubric criteria. They applied the rubric to score student work for a peer-review essay assignment in the ''Information Literacy for Evidence-Based Practice'' course. To determine inter-rater reliability, the raters participated in a follow-up exercise in which they independently applied the rubric to ten samples of work from a research project in the doctor of physical therapy program: the patient case report assignment.Results: For the peer-review essay, a high level of consistency in scoring was achieved for the second workshop, with statistically significant intra-class correlation coefficients above 0.8 for 3 criteria: ''Determine the extent of evidence needed,'' ''Use evidence effectively to accomplish a specific purpose,'' and ''Access the needed evidence.'' Participants concurred that the essay prompt and rubric criteria adequately discriminated the quality of student work for the peer-review essay assignment. When raters independently scored the patient case report assignment, inter-rater agreement was low and statistically insignificant for all rubric criteria (kappa¼À0.16, p.0.05-kappa¼0.12, p.0.05).Conclusions: While the peer-review essay assignment lent itself well to rubric calibration, scorers had a difficult time with the patient case report. Lack of familiarity among some raters with the specifics of the patient case report assignment and subject matter might have accounted for low inter-rater reliability. When norming, it is important to hold conversations about search strategies and expectations of performance. Overall, the authors found the rubric to be appropriate for assessing information literacy skills of graduate health sciences students.

Assessment of knowledge and skills in information literacy instruction for rehabilitation sciences students: a scoping review

“…The course products varied widely; for example, there were 7 searching assignments, where the search strategy was the main focus; 5 case-based inquiry [ 86 ] assignments, where students applied the 5 EBP steps to a patient scenario; 5 critically appraised topic assignments, where students summarized the evidence on a given topic; and 4 research project assignments. One study adapted the Valid Assessment of Learning in Undergraduate Education (VALUE) rubric to assess an EBP assignment [ 82 ].…”

“…Grant and Brettle as well as Evenson used simple yes/no rubrics [ 56 , 57 ]. The adaptation of the VALUE rubric to the graduate physical therapy student context by Turbow and Evener might be the most useful [ 82 ], as it assessed multiple skills in the EBP cycle and could be widely adopted by librarians who are grading case-based inquiries, critically appraised topics, or research project assignments. However, librarians using the VALUE rubric would most likely need to create more detailed requirements for the points awarded for each category in order to make it an effective assessment tool.…”

“…Librarians could work with instructors to integrate the entire adapted or Modified Fresno Test into the course assessment. Another option would be for librarians to use the VALUE rubric as modified by Turbow and Evener [ 82 ] with existing assignments.…”

Students’ Perceptions of Instructional Rubrics in Neurological Physical Therapy and Their Effects on Students’ Engagement and Course Satisfaction

“…Other studies have analyzed the validity and usefulness of various rubrics that assess the information literacy skills of graduate and postgraduate health sciences students, including PT students. Turbow and Evener [ 56 ] found that a modified version of the information literacy Valid Assessment of Learning in Undergraduate Education (VALUE) is appropriate for assessing the information literacy skills of graduate health sciences students, although they also highlight its low interrater reliability in grading clinical case reports. Turbow et al [ 57 ] reach similar conclusions about the usefulness of an adaptation of the VALUE written communication rubric.…”

Documenting the Value of Librarians in the Classroom: Results from a Mixed-Methods Research Collaboration with Campus Partners

“…Additionally, the rubric has been shown to be effective in assessing IL skills. 40 Data merged with Experiences with IL survey module results, and VALUE rubric scores were obtained from the university data warehouse. High school GPA was obtained from transcripts at the point of admission.…”