Deforestation is recognized as a major driver of the loss of biodiversity and ecosystem services. It also disturbs natural processes such as biogeochemical, hydrological, and ecological cycles. In Malawi, deforestation is estimated to be responsible for the loss of 33,000 hectares per year, and is mainly attributed to agriculture expansion, tobacco growing, and excessive use of biomass. However, little research has been conducted at either the local level or that of forests located on customary land. This research aimed to identify and analyze the underlying driving factors associated with the proximate factors of agriculture expansion, tobacco growing, and brick burning in Mwazisi. Landsat images for 1991, 2004, and 2017 were downloaded from the United States Geological Survey website and used to analyze changes in forest cover. Interviews with households (n = 399) and Natural Resource Committee members, a focus group discussion with key officers, and observations were conducted during field data collection in 2017. The results of the land cover analysis showed that forest covered 66% of the study area in 1991, and by 2017 it had decreased to 45.8%. Most households depend on wood from customary land forests for tobacco curing (69%) and brick burning (68%). Furthermore, 47.6% of the households have expanded their agriculture land by approximately 0.57 hectares during the past 15 years. The interview survey and the focus group discussion identified that the underlying driving factors towards these anthropogenic activities are: (a) population growth, (b) poverty, (c) expensive alternative building materials, (d) lack of awareness, (e) lack of resources, (f) lack of commitment from the tobacco companies, and (g) market system of the cash crops grown in the area. In conclusion, a set of economic, institutional, social, and demographic factors, which are associated with imbalanced relationship between rural and urban areas, underpin agriculture expansion, tobacco growing, and brick burning, and have thereby contributed to the decline of the forest cover in Mwazisi, Malawi.