Researchers and developers of learning analytics (LA) systems are increasingly adopting human-centred design (HCD) approaches, with growing need to understand how to apply design practice in different educational settings. In this paper, we present a design narrative of our experience developing dashboards to support middle school mathematics teachers’ pedagogical practices, in a multi-university, multi-school district, improvement science initiative in the United States. Through documentation of our design experience, we offer ways to adapt common HCD methods — contextual design and design tensions — when developing visual analytics systems for educators. We also illuminate how adopting these design methods within the context of improvement science and research–practice partnerships fundamentally influences the design choices we make and the focal questions we undertake. The results of this design process flow naturally from the appropriation and repurposing of tools by district partners and directly inform improvement goals.

With the spread of learning analytics (LA) dashboards in K--12 schools, educators are increasingly expected to make sense of data to inform instruction. However, numerous features of school settings, such as specialized vantage points of educators, may lead to different ways of looking at data. This observation motivates the need to carefully observe and account for the ways data sensemaking occurs, and how it may differ across K--12 professional roles. Our mixed-methods study reports on interviews and think-aloud sessions with middle-school mathematics teachers and instructional coaches from four districts in the United States. By exposing educators to an LA dashboard, we map their varied reactions to visual data and reveal prevalent sensemaking patterns. We find that emotional, analytical, and intentional responses inform educators’ sensemaking and that different roles at the school afford unique vantage points toward data. Based on these findings, we offer a typology for representing sensemaking in a K--12 school context and reflect on how to expand visual LA process models.

The modeling of epistemic knowledge is a necessity of most systems dealing with some sort of artificial reasoning. There a r e several formalisms able to mathematically model someone's degrees of belief. A very popular one is the Bayesian Theory, which is based on a prior knowledge of a probability distribution. Another model is the Theory of Evidence, or DempskrShafer Theory, which provides a method for combining evidences from different sources without prior knowledge of their distributions. In this latter method, it is possible to assign probability values to sets of possibilities rather than to single events only, and it is not needed to divide all the probability values among the events, once the remaining probability should be assigned to the environment and not to the remaining events, thus modeling more naturally certain classes of problems. There are some pitfalls however, in particular, the Dempster-Shafer Theory does not model well evidences with a high degree of conflict, and evidences with the more probable possibility disjoint but with a less probable possibility in common tend to bias the results bward the less probable hypothesis in an illogical way, assigning 100% probability to it.In this paper we present an extension of DempsterShafer Theory that overcome the afore mentioned pitfalls, allowing the combination of evidences with higher degrees of conflict, and avoiding the excessive tendency toward the common possibility of otherwise disjoint hypothesis. This is accomplished by means of a new rule of evidences combination that embodies the conflict among the evidences, modeling naturally the epistemic reasoning.

No abstract