The tumor stroma is a tissue composed primarily of extracellular matrix, fibroblasts, immune cells, and vasculature. Its structure and functions, such as nutrient support and waste removal, are altered during malignancy. Tumor cells transform the fibroblasts into cancer-associated fibroblasts, which have an important immunosuppressive activity, on which growth, invasion and metastasis depend. These activated fibroblasts appear to prevent immune cell infiltration into the tumor nest, thereby promoting cancer progression and inhibiting T cell-based immunotherapy. To better understand the biophysics of the tumor stroma and predict the evolution of cancer cells, we measure the density of different cell types in the stroma using immunohistochemistry stained tumor samples from lung cancer patients. We then incorporate these data, as well as known information on cell proliferation rates and relevant biochemical interactions, into a minimal biomechanical model. We quantify the complex dynamics between species as a function of the system properties, highlighting the inefficiency of immune cells and the fundamental role of activated fibroblasts. A spatio-temporal approach of the inhomogeneous environment and non-uniform cell distributions explains the fate of lung carcinomas. The model reproduces that, while cancer-associated fibroblasts act as a barrier to tumor growth, they also reduce the efficiency of the immune response. Our conclusion is that number of outcomes exist as a result of the competition between the characteristic times of cancer cell growth and the activity rates of the other species. For example, simulations reveal scenarios where tumor nests persist despite the presence of an efficient immune response.