The purpose of this study was to investigate the current status of metal pollution in the sediment from rivers, lakes, and streams in active gold mining districts in Ghana. Two hundred and fifty surface sediment samples from 99 locations were collected and analyzed for concentrations of As, Hg, Cr, Co, Cu, Fe, Zn, Pb, Cd, Ni, and Mn using inductively coupled plasma-mass spectroscopy (ICP-MS). Metal concentrations were then used to assess the human health risks to resident children and adults in central tendency exposure (CTE) and reasonable maximum exposure (RME) scenarios. The concentrations of Pb, Cd, and As were almost twice the threshold values established by the Hong Kong Interim Sediment Quality Guidelines (ISQG). Hg, Cu, and Cr concentrations in sediment were 14, 20, and 26 times higher than the Canadian Freshwater Sediment Guidelines for these elements. Also, the concentrations of Pb, Cu, Cr, and Hg were 3, 11, 12, and 16 times more than the Australian and New Zealand Environment and Conservation Council (ANZECC) sediment guideline values. The results of the human health risk assessment indicate that for ingestion of sediment under the central tendency exposure (CTE) scenario, the cancer risks for child and adult residents from exposure to As were 4.18 × 10−6 and 1.84 × 10−7, respectively. This suggests that up to 4 children out of one million equally exposed children would contract cancer if exposed continuously to As over 70 years (the assumed lifetime). The hazard index for child residents following exposure to Cr(VI) in the RME scenario was 4.2. This is greater than the United States Environmental Protection Agency (USEPA) threshold of 1, indicating that adverse health effects to children from exposure to Cr(VI) are possible. This study demonstrates the urgent need to control industrial emissions and the severe heavy metal pollution in gold mining environments.