Cacao is an important agricultural crop in Ghana, but is grown even in artisanal small scale gold mining (ASGM) area using mercury (Hg) during the amalgamation process. This study investigated Hg concentration in agricultural soil and cacao plant in three ASGM communities in the Ashanti Region of Ghana to evaluate transfer of Hg from soil to cacao and health risk by ingesting of cacao beans. As for the soil samples, the mean Hg concentration was 76 ± 36 µg/kg dw (n = 42), corresponding to moderate contamination by the contamination factor. As for the cacao samples (n = 12 each), the highest mean Hg concentration was found in roots (211 µg/kg dw) and followed by leaves (138 µg/kg dw), pods (40 µg/kg dw) and beans (27 µg/kg dw). Significant correlations were observed between the cacao samples (p < 0.05), but not with the soil samples. indicating accumulation does not only depend on contaminated soil but also the type of plant, its ability to absorb and atmospheric deposition. Data analysis using the bioaccumulation factor (BAF) and the transfer factor (TF) indicated that Hg was highly accumulated from the soil to the root of cacao (BAF = 3.1) but did not highly transfer from the root to the aerial parts (TF < 1). The health risk assessment for ingestion of cacao beans showed that the hazard quotient was less than 1, indicating that there is no possibility of non-carcinogenic health effect.