Predictive algorithms and racial bias: a qualitative descriptive study on the perceptions of algorithm accuracy in higher education

Abstract

Purpose This study aims to explore the perception of algorithm accuracy among data professionals in higher education. Design/methodology/approach Social justice theory guided the qualitative descriptive study and emphasized four principles: access, participation, equity and human rights. Data collection included eight online open-ended questionnaires and six semi-structured interviews. Participants included higher education professionals who have worked with predictive algorithm (PA) recommendations programmed with student data. Findings Participants are aware of systemic and racial bias in their PA inputs and outputs and acknowledge their responsibility to ethically use PA recommendations with students in historically underrepresented groups (HUGs). For some participants, examining these topics through the lens of social justice was a new experience, which caused them to look at PAs in new ways. Research limitations/implications Small sample size is a limitation of the study. Implications for practice include increased stakeholder training, creating an ethical data strategy that protects students, incorporating adverse childhood experiences data with algorithm recommendations, and applying a modified critical race theory framework to algorithm outputs. Originality/value The study explored the perception of algorithm accuracy among data professionals in higher education. Examining this topic through a social justice lens contributes to limited research in the field. It also presents implications for addressing racial bias when using PAs with students in HUGs.

Guest editorial: The datafication of student life in higher education: privacy problems and paths forward

“…Von Winckelmann (2023) “Predictive algorithms and racial bias: a qualitative descriptive study on the perceptions of algorithm accuracy in higher education”, Information and Learning Sciences , doi: 10.1108/ILS-05-2023-0045:…”

“…Noting that predictive algorithms “have become the most common analytic tool used in higher education […] and open a window into the educational lives of students,” Von Winckelmann (2023) catalogs how higher education institutions have used these tools to investigate and improve student success and engagement, support alumni fundraising strategies, identify students likely to default on their student loans and target students whose timeline to graduation is slower than institutionally expected. Like with uses of predictive algorithms in other contexts, they warn that inappropriate uses of predictive algorithms “places students in historically underrepresented groups (HUGs) in a precarious position as there are significant risks of racial biases infiltrating the data.” Von Winckelmann (2023) used a questionnaire and interview protocol informed by data justice theory to investigate how higher education data professionals perceive and vet the accuracy of the algorithms their institutions use. The study confirmed that participants were “aware of both systemic and racial bias in their [predictive algorithm] inputs and outputs and acknowledge their responsibility to use [predictive algorithms] recommendations ethically with students in HUGs.” Among other findings, resulting practical implications from the study recommend that higher education data professionals would be well served by social justice professional education related to data practices.…”

“…Each article uniquely presents pragmatic insights into how practitioners, researchers, administrators and even technologists could affect positive change in their uses of educational technologies and student data to either reduce privacy concerns or enhance privacy protections. The articles suggest frameworks for informing data practices (Von Winckelmann, 2023), the power of existing governance norms to fill governance gaps (Sanfilippo et al , 2023), a clear call to action to create and update privacy policies in response to sociotechnical changes (Mann et al , 2023), advancements in privacy-protecting technologies that support research endeavors (Holmes et al , 2023), new ways of thinking about student behaviors and attitudes toward learning technologies (Greenhalgh et al , 2023) and a concrete definition and actionable conceptualizations of trust that can drive how LA can be implemented in trustworthy ways. There is always room for theorizing and philosophizing about the concept of privacy and its value – especially in relation to sociotechnical change – but privacy must also be practiced.…”