Propolis is known as a source of compounds with strong antibacterial activity. Due to the antibacterial effect against streptococci of the oral cavity, it seems to be a useful agent in decreasing the accumulation of dental plaque. It is rich in polyphenols which are responsible for a beneficial impact on the oral microbiota and antibacterial effect. The aim of the study was to evaluate the antibacterial effect of Polish propolis against cariogenic bacteria. The minimum inhibitory concentration (MIC) and minimum bactericidal concentration (MBC) were determined on cariogenic streptococci related to the occurrence of dental caries. Lozenges based on xylitol, glycerin, gelatin, water, and ethanol extract of propolis (EEP) were prepared. The effect of prepared lozenges on cariogenic bacteria was assessed. Propolis was compared to chlorhexidine which is used in dentistry as the gold standard. In addition, the prepared propolis formulation was stored under stress conditions to assess the influence of physical conditions (i.e., temperature, relative humidity, and UV radiation). In the experiment, thermal analyses were also performed to evaluate the compatibility of propolis with the substrate used to create the base of lozenges. The observed antibacterial effect of propolis and prepared lozenges with EEP may suggest directing subsequent research on prophylactic and therapeutic properties decreasing the accumulation of dental plaque. Therefore, it is worth highlighting that propolis may play an important role in the management of dental health and bring advantages in preventing periodontal diseases and caries as well as dental plaque. The colorimetric analyses carried out in the CIE L*a*b* system, microscopic examinations, and TGA/DTG/c-DTA measurements indicate the unfavorable effect of the tested storage conditions on the lozenges with propolis. This fact is particularly evident for lozenges stored under stress conditions, i.e., 40 °C/75% RH/14 days, and lozenges exposed to UVA radiation for 60 min. In addition, the obtained thermograms of the tested samples indicate the thermal compatibility of the ingredients used to create the formulation of lozenges.