Freeze-dried immobilized Lactobacillus casei ATCC 393 on casein and apple pieces were assessed as a probiotic adjunct culture for novel probiotic yogurt production. The effect of probiotic culture on physicochemical characteristics, probiotic cell survival, volatile aroma compounds, and sensory quality were evaluated during 28 days of storage at 4 °C. The use of L. casei resulted in lower pH values (3.92–4.12), higher acidity (0.88–1.10 g lactic acid/100 g of yogurt), and lower syneresis (40.8%–42.6%) compared to traditionally produced yogurt (pH 4.29; acidity 0.83 g lactic acid/100 g of yogurt; syneresis 44.1%). Microbiological and strain-specific multiplex PCR (Polymerase Chain Reaction) analysis confirmed that immobilized L. casei ATCC 393 cells were detected in yogurts at levels >7 log cfu g−1 after 28 days. In addition, probiotic supplementation significantly affected the concentrations of key volatile compounds, like acetic and other organic acids, 2-ethyl-1-hexanol, acetoin, and 2-butanone, as revealed by GC-MS (Gas Chromatography–Mass Spectrometry) analysis. Finally, the sensory evaluation demonstrated that the new products exhibited improved characteristics compared to traditionally produced yogurts.