Nonoxide ceramics excel among the reinforcements used for aluminum matrix composites due to their variety of morphologies and mechanical properties. Among these reinforcements are carbides (SiC, B4C, and WC); carbon materials (graphite, carbon fibers, carbon nanotubes, and graphene); nitrides (silicon nitride [Si3N4] and BN); and hollow Fe spheres. Although the effect of adding different percentages of reinforcements has been widely studied for Al matrices, matrix–reinforcement interactions need more attention. The consequences of these interactions can include interface formation, loss of alloying elements, reinforcement deterioration, modifications in the matrix microstructure, different precipitation sequences and kinetics, and interfacial diffusion of elements. These interactions may be significantly modified by the alloying elements, needing more in-depth analyses for a correct selection of the matrix–reinforcement system. Al matrices with Si, Cu, and Mg outstand, and the focus of the present work is their reciprocal interactions with nonoxide reinforcements. The novelty of this review consists of the analysis and discussion of these interactions, emphasizing the modifications originated by each one of these alloying elements, and the conditions needed to increase or avoid their effects on the composite. Besides, an analysis of the crystallography of the generated interfaces is presented, including their impact on mechanical properties. This could be helpful for a better understanding and selection of the matrix–reinforcement system, also serving as a benchmark study.