In this paper, we present a mathematical model that analyzes cancer treatment via radiotherapy and proposes ways to control its progression. To identify preventive measures for cancer in its early stages, scientists have studied both risk factors and protective factors. In our model, we consider the disease‐free equilibrium points, namely, the trivial equilibrium (TE), healthy cell absenteeism equilibrium (HCAE), cancer cell absenteeism equilibrium (CCAE), and cancer cell incidence equilibrium (CCIE). We use nonlinear analysis techniques and the spectral radius method to study the stability and instability of systems. Since the reproduction number plays a critical role in stability analysis, we use the spectral radius method to evaluate it. We have also added a control function to the model to enhance it by including interactions between healthy and cancer cells. Optimization techniques are used to identify the limitations and needs of the problem and derive the best solutions to control the tumor. We conduct sensitivity analysis with respect to various parameters to study the model's robustness. To validate our methods and results, we provide simulations and numerical analysis.