The study of raw materials is an essential step in lithic analysis, regardless of the age, provenance, and technology of the assemblages. As in many other contexts of the Paleolithic, researchers of the Middle Stone Age (MSA) in southern Africa have often focused their attention on fine-grained, non-local rock types, such as silcrete. Here, I spotlight raw materials considered to be of lower suitability for knapping and frequently acquired from local sources. Due to their coarse-grained nature, artifacts from rock types such as calcrete, sandstone, and quartzite might show attributes that are different from finer-grained materials. Some of these knapped stones even constitute the substrate of the sites they are from, at times resulting in their neglect or not being recognized as anthropogenic artifacts. Knapped vein quartz features sharp and durable edges, but its complicated fracture mechanics hamper comparative analysis and provide methodological challenges. In this study, raw materials from different transport distances and with different presumed qualities are compared in terms of their roles in MSA lithic technology and settlement patterns. In the first step, the article focuses on the open-air special-purpose camp of Hoedjiespunt 1 (HDP1, Western Cape) and the rockshelter residential site of Sibudu (KwaZulu-Natal), especially on assemblages dated between ~ 130–100 and ~ 58 ka. Subsequently, I review relevant materials for the southern African MSA. At HDP1 and Sibudu, local raw materials of lower knapping suitability assume several roles, from the “staple” material for all manufacturing stages to special-purpose and “add-on” functions. In the broader southern African region, MSA knappers also used these rock types in a flexible manner with gradual differences but also similarities to their use of finer-grained raw material. These differences depend on a complex interaction of raw material availability, differential site use, and the position of the localities in the settlement system.