No abstract

Because human processes are subtle, complex, and contextualized, computational representations of those processes face highly significant unmet design challenges. Design Thinking (DT) offers a potential new paradigm of creativity and innovation in education capable of effecting meaningful culture change. DT is nonlinear but encompasses elements of empathy, problem definition, ideation, prototyping, and tests that may freely move as needed from and to each other. DT's empathic focus on end users' needs suggests educational measurement's information infrastructures will have to coherently integrate assessment and instruction across multiple levels of complexity in communication. Applying DT reveals the need to attend to previously undeveloped technical issues in communication. Especially important are developmental, horizontal, and vertical forms of coherence, and denotative, metalinguistic, and metacommunicative levels of complexity. New solutions emerge when classrooms are reconceived as meta-design ecosystem niches of creativity and innovation structured from the bottom up by flows of self-organizing information. Recently identified correspondences between educational measurement and metrology support efforts aimed at developing multilevel common languages for the communication of learning outcomes. Prototype reports illustrate how emergent measured constructs can be brought into language in ways that integrate developmental, horizontal, and vertical coherence across levels of complexity. Coherent information infrastructures of these kinds are capable of adapting to new circumstances as populations of persons and items change, doing so without compromising the continuity of comparisons or the uniqueness of locally situated knowledge and practices.

Assessment for learning aims to improve learning by feeding back information on the attainment of intended learning objectives to students and teachers. In this form of assessment, the student acts as a measuring instrument and learning attainment is the quantity to be measured. Current practices in classroom assessment, unfortunately, often are not grounded and have been focused on total correct scores without reference to the evidence hidden within them concerning the attainment of learning objectives. Our goal in this study is to clarify what assessment for learning is, why it is important, and how it works in a metrologically coherent context of equated assessments administered from calibrated item banks. The main focus lies in how item content can be framed according to intended learning outcomes that are meaningfully interpreted using a construct map. An example involves empirical datasets of 283 secondary three students from two formative tests designed by two science teachers. Coherent formative assessment of this kind applies a scientific model that belongs to the same class metrologists consider as defining measurement. The results illuminate the potential for establishing item banks that enable teachers to more efficiently and effectively assess and improve the attainment of learning objectives across different cohorts of students in a common metric.

Student Evaluations of Teaching (SET) have been adopted worldwide as a standard practice to enhance teaching and learning quality. However, improvement efforts are threatened by various problems involving ordinal scores and interpretive frameworks restricted to individual item sets and uninformed by a well-defined invariant construct. In this study, an approach to framing SET items according to theory rather than personal or subjective decisions is described. A sample of 920 students from the University of Macau participating in this study provided responses to 42 SET items. Applying a scientific model of measurement, the data collected were analysed and interpreted. Justification between theoretical and measurement analysis informed the creation of a theoretical SET construct map. That map and the associated item hierarchy were used as reference sources for configuring SET reports and recommendations. A new coherent paradigm for SET documentation is proposed for further study.

No abstract