Malaria is a major health concern for many developing countries. Designing strategies for efficient distribution of malaria medications, such as Artemesinin Combination Therapies, is a key challenge in resource constrained countries. This paper develops a solution methodology that integrates strategic‐level and tactical‐level models to better manage pharmaceutical distribution through a three‐tier centralized health system, which is common to sub‐Saharan African countries. At the strategic level, we develop a two‐stage stochastic programming approach to address the problem of demand uncertainty. In the first stage, an initial round of shipments is sent before the malaria season to each local clinic from district hospitals, which receive medications from regional warehouses. After the malaria season begins, a recourse action is triggered to avoid shortages in the form of (i) lateral transshipment or (ii) delayed shipment. The optimal solutions developed by the strategic model identify small clinic clusters possessing exclusive transshipment policies. Therefore, we decompose the problem at the tactical level, solving each clinic cluster independently using a Markov decision process approach to determine optimal periodic transshipment policies. A case study of our proposed distribution system is performed for 290 facilities controlled by the Malawi Ministry of Health. Numerical analysis of Malawi's distribution system indicates that our proposed cluster‐based decomposition method could near optimally reduce shortage incidents. Moreover, such an approach is robust to challenges of developing countries such as slow paper‐based inventory review, uncertain transportation infrastructure, the need for equitable distribution, and seasonal and correlated demand associated with malaria transmission dynamics.