This non-experimental, exploratory and descriptive study, using a qualitative case study approach, aims to investigate whether there is evidence of variance in the quality of school-based assessment (SBA) in Grade 9 mathematics. Participants were purposefully selected from five schools in a district in the Northern Cape in South Africa. After questionnaires were completed, individual face-to-face semi-structured interviews were conducted with participants from the participating schools. Documents were collected and analysed to corroborate or contradict data from the questionnaires and interviews. Lack of adherence to policy, variation in classroom practice and inconsistent monitoring and moderation practices were identified as themes of possible sources of variation in SBA. An analysis of the interviews and document analysis revealed that most of the Heads of Department and principals lacked in-depth knowledge and understanding of their roles and functions in making SBA reliable, credible and valid. This was not only due to a lack of capacity to perform such functions, but was also due to a lack of effective induction and training by the district and provincial offices. Findings from the current study point to the necessary role that a periodic evaluation of SBA may play to ensure its effectiveness, credibility and reliability as part of successful assessment practices in a mostly developing context.