The presence of high levels of arsenic (As) in rice fields has negative effects on the health of those consuming rice as their subsistence food. This study determined the variation in total As concentration in local aromatic rice (LAR) (kalijira) and two high-yielding varieties (HYVs) (BRRI dhan 32 and BRRI dhan 28) grown in paddy fields in Matlab, Bangladesh, an As hotspot with elevated As levels in groundwater. Mature rice grain samples and soil samples were collected from different paddy fields, and the As concentrations in both the de-husked grains and the husks of the three rice cultivars were analysed to identify the safest of the three cultivars for human consumption. The results showed that the total As concentration was higher (0.09–0.21 mg As kg−1) in the de-husked grains of LAR than in the husks, while the opposite was found for the HYV rice. Moreover, the As concentration in soil samples was 2 to 5-fold higher for the LAR than for the HYVs, but the As accumulation factor (AF) was lower in the LAR (0.2–0.4%) than in the HYVs (0.9–1%). Thus, LAR can be considered the safest of the three cultivars for human consumption owing to its low AF value. Furthermore, due to the low AF, growing LAR instead of HYVs in soils with slightly elevated As levels could help improve the food safety level in the food chain.