Background High price is a major challenge limiting access to essential medicines especially among the poorest families in developing countries. The study aims to compare the prices of medicines used in the management of pain, diabetes, and cardiovascular diseases in private pharmacies and the National Health Insurance Fund (NHIF) in Tanzania. Pharmacy prices were also compared with the prices of medicines surveyed nationally by WHO/HAI in 2012. Method This cross-sectional study was conducted in Dar es Salaam, Morogoro, Dodoma, and Kilimanjaro regions from February to April 2015. Data were collected from 33 private pharmacies, NHIF and, the HAI database. The study used the WHO/HAI methodology. The analysis was done using non-parametric Kruskal-Wallis and post-hoc pair-wise comparison Dunn test, while a possible change in prices between our survey and 2012 WHO/HAI national survey data was tested using a Sign test in Stata version 16.1. Results Twenty-eight essential medicines, of which 9 are used for management of pain, 7 for diabetes, and 12 for cardiovascular diseases were analyzed. There was a significant difference in the mean pharmacy prices of some medicines between the regions and between the pharmacies and NHIF reference prices. NHIF reference prices were higher than the pharmacy prices for 16 of the 28 medicines. There was a significant increase in the prices of 5 out of the 8 medicines that were also nationally surveyed by the WHO/HAI in 2012. Conclusion The study found that medicine prices in private pharmacies vary a lot between the study regions, which raises equity concerns. Also, there was a significant difference between the pharmacy and the NHIF reimbursement prices, which may expose patients to fraudulent co-payments or hinder timely access to prescribed medicines. Therefore, effective price control policies and regulations for medicines are warranted in Tanzania.