Instruction and assessment of information literacy among STEM majors

Abstract

The ability to find, use, and communicate information effectively is a critical competency for 21st century engineers. Thus it is an important element in retention, graduation, and placement of professional engineers.This study found scaffolded, engaging instruction resulted in impressive gains in higher-order skills associated with writing and information literacy among upper division engineering undergraduates. Students were evaluated using a brief diagnostic essay as an assignment and as a pre-and posttest to measure these skills in engineering majors taking a required technical writing course. The instruction included using Wikipedia as both a teaching tool and a writing assignment. The instrument was scored by multiple readers using a rubric. Interrater reliability was high and validation showed that the instrument could reliably measure and discriminate among five unique information literacy variables. Use of this constructedresponse diagnostic enables the evaluation of higher order research and writing skills that have been difficult to assess using limited-response tests, so that learning experiences may be better designed to reach students at the appropriate skill level.

Evaluation of competency methods in engineering education: a systematic review

“…Rubrics, scoring methods with or without detailed descriptions of levels of performance, were used by faculty (Gerlick et al 2011) or industry representatives (Hotaling et al 2012) to assess written reports and oral presentations, designs projects, and capstone courses. Rubrics with detailed descriptions of levels of performance homogenised and guided the assessors (Flateby and Fehr 2008;Scharf 2014;Eichelman, Clark, and Bodnar 2015) because they increased inter-rater reliability and minimised subjectivity of the competency measurement process (Fila and Purzer 2012).…”

“…Rubrics can be used as a checklist to verify whether students demonstrate the pre-defined competencies and at which level (Fila and Purzer 2012). When rubrics are objectively created and validated to measure students' behaviours, they are great measurement methods that improve inter-marker consistency and reduce marker bias effects (Flateby and Fehr 2008;Scharf 2014;Eichelman, Clark, and Bodnar 2015). In addition, this consistency can be optimised with the use of more than one rater or grader and the standardisation of the scales according to graders' scores (Ward, Gruppen, and Regehr 2002).…”

Continuing the Conversation: Questions about the Framework

“…We see this broad acceptance of the Standards and its vocabulary over the past 25 years as a persuasive reason to retain them even while enriching them with the benefit of those years and with some of the new viewpoints presented in the Framework. (Charles, 2015;Hsieh & Holden, 2010;Hsieh, Dawson, Hofmann, Titus, & Carlin, 2014;Scharf, 2014 While initially discussing how to use the Framework in our instruction programs, the authors questioned whether the Framework could be used for the assessment of student learning. Although the ACRL Task Force included sample lessons and assignments in the earliest drafts of the Framework, those were removed from the final document with the expectation that the profession would experiment, create, and share ideas in a sandbox or repository (Gibson, Carbery, Hensley, Miller & DiNardo, 2015, 56:20-57:14).…”

Understanding Undergraduate Engineering Student Information Access and Needs: Results from a Scoping Review

“…( Figure 1, Table 1). Of the 44 papers included quantitative [12], [20], [25], [30], [40]- [42], [46], [49], [50], [52], [54], [56]- [58], [63]; 14 qualitative [27]- [29], [33], [36], [37], [44], [45], [51], [53], [59], [61], [62], [65]; 11 mixed methods studies [18], [26], [31], [32], [38], [39], [43], [47], [48], [60], [66]; two literature reviews [34], [35]; and one instrument development paper [55]. The included studies were published between 1991 and 2018, having a median age of 5.5 years, and with five of the studies [28], [31], [36], [41],…”

“…Papers were primarily published in the United States (n=26) [12], [20], [27]- [30], [33]- [36], [38]- [43], [45], [47], [53]- [55], [58], [59], [61]- [63], the remaining papers were published in Canada (n=7) [12,[25][26][27][28][29][30], Australia (n=4) [20][21][22][23], Botswana (n=1) [51], United Kingdom (n=1) [37], Kuwait (n=1) [25], Ireland (n=1) [44], India (n=1) [56] and Singapore (n=1) [50]. As indicated in Table 2, the majority of studies use a survey as a part of their methodology (n=24) [12], [18], [20], [25], [26], [30], [31], [38]- [43], [46]- [50], [52], [54]…”