In this paper we develop an approach to modeling and simulating the process of infection transmission among individuals and the effectiveness of protective counter-measures. We base our approach on pedestrian dynamics and we implement it as an extension of the Vadere simulation framework. In order to enable a convenient simulation process for a variety of scenarios, we allow the user to interact with the simulated virtual environment (VE) during run time, for example, by dynamically opening/closing doors for room ventilation and moving/stopping agents for re-positioning their locations. We calibrate and evaluate our approach on a real-life case study—simulating COVID-19 infection transmission in two kinds of scenarios: large-scale (such as the city of Münster, Germany) and small-scale (such as the most common indoor environments—classrooms, restaurants, etc.). By using the tunable parameters of our modeling approach, we can simulate and predict the effectiveness of specific anti-COVID protective measures, such as social distancing, wearing masks, self-isolation, schools closing, etc.