Technology enhanced assessment in complex collaborative settings

Abstract

Building upon discussions by the Assessment Working Group at EDUsummIT 2013, this article reviews recent developments in technology enabled assessments of collaborative problem solving in order to point out where computerised assessments are particularly useful (and where noncomputerised assessments need to be retained or developed) while assuring that the purposes and designs are transparent and empowering for teachers and learners. Technology enabled assessments of higher order critical thinking in a collaborative social context can provide data about the actions, communications and products created by a learner in a designed task space. Principled assessment design is required in order for such a space to provide trustworthy evidence of learning, and the design must incorporate and take account of the engagement of the audiences for the assessment as well as vary with the purposes and contexts of the assessment. Technology enhanced assessment enables in-depth unobtrusive documentation or 'quiet assessment' of the many layers and dynamics of authentic performance and allows greater flexibility and dynamic interactions in and among the design features. Most important for assessment FOR learning, are interactive features that allow the learner to turn up or down the intensity, amount and sharpness of the information needed for self-absorption and adoption of the feedback. Most important in assessment OF learning, are features that compare the learner with external standards of performance. Most important in assessment AS learning, are features that allow multiple performances and a wide array of affordances for authentic action, communication and the production of artefacts.

Computer science in K-12 school curricula of the 2lst century: Why, what and when?

“…IFIP discussions generally agreed that, whereas these intellectual practices and social competences are very important in learning Computer Science, the challenges for their assessment can lead to their under-emphasis. This problem is not only evident in learning Computer Science but more generally in curricula (Webb and Gibson 2015).…”

“…In this way the creative aspects of Computer Science, for example, may become lost from a curriculum because they are not easily assessed through traditional exams and this concern was expressed by many who attended the panel. Opportunities provided by new computer-based assessments may help to solve this problem but significant research and development challenges remain (see for example Webb and Gibson 2015).…”

Data science in educational assessment

“…The paper aims to extend our analysis of assessments or evaluations of collaborative problem solving (CPS) as implemented by the OECD PISA started in Webb and Gibson (2015) to examine the significance of big data for this assessment approach and then for educational assessment more broadly. In the companion article (ibid.…”

“…Until recently, the field dealt almost exclusively with the construction and validation of measurement instruments such as questionnaires, tests, and personality assessments. However there is now a need to expand to include highly interactive digital learning and adaptive test experiences, such as the OECD PISA assessment of CPS discussed in the companion article (Webb and Gibson 2015). In brief, PISA is a triennial international survey that aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students in order to determine the extent to which they can apply their knowledge to real-life situations and hence are prepared for full participation in society.…”

Assessing Collaborative Problem Solving Skills in Technology-Enhanced Learning Environments – The PISA Framework and Modes of Communication

“…While the PISA framework have been refereed to as a valued departure point for assessment of CPS by other researchers [19][15], the research literature lacks papers reporting on empirical uses and evaluations of the framework. Therefore, we currently, on the one hand, lack understanding of how the PISA assessment framework can be utilized, and on the other hand, of the actual merits of the framework in assessing students' collaborative problem solving skills in technology enhanced assessment environments.…”