With a wide range of etiologies, low back pain (LBP) presents a true clinical challenge, finding its origins both in intrinsic spinal and systemic conditions, as well as referred ones. This review categorizes the LBP into these three groups and aims to offer a comprehensive look at the tools required to diagnose and differentiate them. The intrinsic etiologies are based on conditions that affect the musculoskeletal components of the lumbar spine, such as intervertebral disc disease, stenosis, muscular imbalance, and facet joint degeneration. The systemic causes usually extend beyond local structures. Such are the cases of neoplasia, infections, and chronic inflammation. The diagnosis is rendered even more complex by adding the referred pain, which only manifests in the lower back yet arises in more distant locations. By synthesizing the literature that encompasses the problem, this review aims to augment the understanding of the differential diagnoses of LBP by showcasing the subject’s nuances. This categorization provides a structured approach to a patient-centered diagnosis, which could facilitate the medical practitioners’ efforts to navigate this pathology more effectively.