Bone strength in human cortical bone is determined by the composition and structure of both the mineral and collagen matrices and influenced by factors such as age, gender, health, lifestyle and genetic factors. Age-related changes in the bone matrix are known to result in loss of mechanical strength and increased fragility. In this study we show how Raman spectroscopy, with its exquisite sensitivity to the molecular structure of bone, reveals new insights into age- and sex-related differences. Raman analysis of 18 samples of cortical hip bone obtained from people aged between 47–82 years with osteoarthritis (OA) found subtle changes in the lipid and collagen secondary structure, and the carbonate (CO32−) and phosphate (PO43−) mineral ratios in the bone matrix. Significant differences were observed between older and younger bones, and between older female and older male bones; no significant differences were observed between younger male and female bones. Older female bones presented the lowest mineral to matrix ratios (MMR) and highest CO32−/PO43− ratios, and relative to lipid/collagen –CH2 deformation modes at 1450 cm−1 they had lowest overall mineral content, higher collagen cross linking and lipid content but lower levels of α-helix collagen structures than older male and younger male and female bones. These observations provided further insight on bone composition changes observed in the bone volume fraction (BV/TV) for the older female bones from microCT measurements on the same samples, while tissue mineral density (TMD) measurements had shown no significant differences between the samples.