Cell-cell interaction networks are pivotal in cancer development and treatment response. These networks can be inferred from data, however this process often combines data from multiple patients, and/or creates networks on a cell-types level. It creates a good average overview of cell-cell interaction networks, but fails to capture patient heterogeneity and/or masks potentially relevant local network structures. We propose a mathematical model based on random graphs (called RaCInG) to alleviate these issues using prior knowledge on potential cellular interactions and patient's bulk RNA-seq data. We have applied RaCInG to extract 444 network features related to the tumor microenvironment, unveiled associations with immune response and subtypes, and identified cancer-type specific differences in inter-cellular signaling. Additionally, we have used RaCInG to explain how immune phenotypes regulated by context-specific intercellular communication affect immunotherapy response. RaCInG is a modular pipeline, and we envision its application for cell-cell interaction reconstruction in different contexts.