Using Direct Information Literacy Assessment to Improve Mechanical Engineering Student Learning - A Report on Rubric Analysis of Student Research Assignments

Abstract

This quasiexperimental study examined the effectiveness of library instruction in a junior level mechanical engineering design process course during the Spring 2013 and Fall 2013 semesters. In the Spring of 2013 librarians delivered an active learning session to students, placing heavy emphasis on the breadth of information resources available for engineering design research beyond what can be found through Google. The session was designed to build on university general education information literacy outcomes. Student research assignments completed after the library session were analyzed using an information literacy rubric. Based on the results, librarians collaborated with the course instructor to modify the Fall 2013 library session to place the heaviest emphasis on search strategy development, rather than the breadth of information resources, and adapt the research assignment requirements. The revised session also aligned with the Searching as Strategic Exploration frame of the ACRL Framework for Information Literacy . Student assignments were again analyzed after the Fall 2013 session. The results showed a positive association between the revised instruction and assignment and rubric scores. Making changes to the library instruction and assignment based on assessment results helped to ensure that librarian and course instructor time spent preparing course assignments and delivering the session material was being spent most efficiently.

Assessing Information Literacy in Capstone Design Projects: Where are students still struggling?

“…The VALUE rubric was used to begin scoring papers, but it rapidly became clear that the generic rubric was not specific enough to the Capstone assignment to be applied accurately. Based on findings from previous studies on rubric-based assessment, a customized version of the VALUE rubric and other sources was designed to more closely align with the Capstone assignment [12], [13], [16], [17]. The customizations included specifying scoring criteria based on the expected outcomes of the assignment, as well as eliminating certain elements that were not included.…”

“…However, relatively few studies have assessed the IL outcomes of Capstone students using rubrics. Of the existing studies that use this method, the most challenging IL skills identified are limited ability to define information needs and getting beyond using search engines for quick fact-finding to identify quality, in-depth research [17]. Students also had trouble with differentiating between different types of sources, lack of assessment of quality reference material, lack of familiarity with how to find and use technical papers, few references, and improper use of citations and what information needs to be cited [16], "usage of references across paper", "consistency of citation style across paper", lack of overall time spent on information gathering, and challenges with evaluating web resources [18].…”

Reinforcing Information Fluency: Instruction Collaboration in Senior Capstone Laboratory Course

“…Using rubrics to assess information fluency skills in engineering students has been used previously. Phillips, Lucchesi, Sams, and van Susante [25] used a rubric to evaluate student bibliographies in a junior-level Engineering Design Processes course. The rubric results led to revisions of librarian instruction to include more focus on search strategies instead of specific source formats.…”

“…The rubric used in the assessment of the students' reports (see Appendix C) was modeled after previous studies [15], [25], as well as the AACU Information Literacy VALUE Rubric [26]. The rubric was used in scoring the quality of sources and ethical use of information in the students' reports for both Lab A and Lab B.…”