Objective: Balneotherapy, a domain of medical science, focuses on utilizing the beneficial effects of medicinal waters. Low back pain is among the most prevalent musculoskeletal disorders affecting a large proportion of the population during their lifetime. Although small in number, all controlled studies published on this subject have demonstrated the benefits of balneotherapy. This present study was undertaken to compare the effects of hydrotherapy with mineral water vs. tap water on low back pain. Patients and Methods: A single-blind clinical study was carried out to appraise the therapeutic efficacy of reduced sulphurous water on 60 patients with low back pain. 30 subjects took baths in reduced sulphurous mineral water, whereas the other 30 patients used modified tap water of matching odor. Parameters determined at baseline, after balneo-/hydrotherapy, and at the end of the 3-month follow-up period included the results of the Visual Analogue Scale (VAS) score, the modified Oswestry index, mobility of the spine, antalgic posture, tenderness of the paravertebral muscles on palpation, the dose requirements for analgesics, and the efficacy assessed by the investigators and by the patients. Results: Bathing in mineral water resulted in a statistically significant improvement. This was reflected by the VAS (p < 0.01) and manifested by the mitigation of muscle spasm (p < 0.01), the alleviation of local tenderness (p < 0.01), the enhanced flexion-extension and rotation of the spine (p < 0.01) as well as by the improvement of the Schober’s index (p < 0.01). All these beneficial changes persisted as long as 3 months after the completion of balneotherapy. By contrast, hydrotherapy with tap water resulted only in the temporary improvement of just a single parameter: the VAS score improved significantly (p < 0.01). Conclusions: Balneotherapy in itself can alleviate low back pain. As demonstrated by this study, the analgesic efficacy and improvement of mobility accomplished by the use of mineral water is significantly superior to that afforded by hydrotherapy with tap water. Our results clearly establish the beneficial effects of mineral water. Moreover, it is a valuable adjunct to other forms of physical treatment as well as to pharmacotherapy.