The aim of this study was to evaluate the applicability of selected Lactobacillus strains, previously isolated from spontaneously fermented foods, as starter cultures in the production of organic dairy products—acid-rennet goat’s cheeses under industrial conditions. The basic composition and the effect of starter cultures on the physicochemical, microbiological, sensory as well textural properties during the production and storage of goat’s cheese were evaluated. Lactic acid bacteria count in cheese samples was at a high level of about 8 log CFU/g. The cheeses made with Levilactobacillus brevis B1 and Lactiplantibacillus plantarum Os2 bacterial cultures additions have showed more favorable Lipid Quality Indices than for the control one with the addition of acid whey. The time of ripening of the cheeses significantly (p < 0.005) changed their consistency—they became softer and more elastic and less moist. It is possible that the selected cultures of L. brevis B1 and L. plantarum Os2 isolated from traditional cheeses can be successfully applied to goat’s milk cheese production. The strain L. brevis B1 is highly recommended as a starter culture for goat’s milk cheese production, taking into account the good microbiological and sensory quality as well as the chemical composition.