Defects dictate the properties of many functional materials. To understand the behaviour of defects and their impact on physical properties, it is necessary to identify the most stable defect geometries. However, global structure searching is computationally challenging for high-throughput defect studies or materials with complex defect landscapes, like alloys or disordered solids. Here, we tackle this limitation by harnessing a machine-learning surrogate model to qualitatively explore the structural landscape of neutral point defects. By learning defect motifs in a family of related metal chalcogenide and mixed anion crystals, the model successfully predicts favourable reconstructions for unseen defects in unseen compositions for 90% of cases, thereby reducing the number of first-principles calculations by 73%. Using CdSexTe1−x alloys as an exemplar, we train a model on the end member compositions and apply it to find the stable geometries of all inequivalent vacancies for a range of mixing concentrations, thus enabling more accurate and faster defect studies for configurationally complex systems.