Drug combinations provide effective treatments for diverse diseases, but also represent a major cause of adverse reactions. Currently there is no systematic understanding of how the complex cellular perturbations induced by different drugs influence each other. Here, we introduce a mathematical framework for classifying any interaction between perturbations with high-dimensional effects into 12 interaction types. We apply our framework to a large-scale imaging screen of cell morphology changes induced by diverse drugs and their combination, resulting in a perturbome network of 242 drugs and 1832 interactions. Our analysis of the chemical and biological features of the drugs reveals distinct molecular fingerprints for each interaction type. We find a direct link between drug similarities on the cell morphology level and the distance of their respective protein targets within the cellular interactome of molecular interactions. The interactome distance is also predictive for different types of drug interactions.