We present a methodology leveraging machine learning models to generate interatomic potentials for crystalline materials. This approach is rooted in the material's crystallography in question. Specifically, we tap into the occupied Wyckoff sites, extracting the defining features that encapsulate the atomic local order in the material. Our choice for the target variable is the formation energy per atom, derived from the total energy of the structure's representative cell. Our machine learning model's architecture depends on the occupied Wyckoff sites. The diversity of these occupied sites conditions the layering scheme within the model. Atoms occupying a particular Wyckoff site were modeled with the architecture and learning parameters linked to the respective layer. To illustrate our method, we developed an interatomic potential for atomic interactions in α-alumina. For training purposes, we generated the samples through quantum mechanical computations. The evaluation of the learned interatomic potential involved conducting molecular dynamics calculations on a 2 × 2 × 2 supercell, yielding formation energies per atom deviating by less than 1.0 meV from the quantum mechanics results. The methodology described here paves the way for further innovations, potentially ushering in the creation of interatomic potentials that can be utilized for more than one material. Moreover, this approach provides valuable insights into the feasibility of substituting atoms within a compound by focusing exclusively on the specific Wyckoff sites that they occupy.