Microfinance in developing areas such as Africa has been proven to improve the local economy significantly. However, many applicants in developing areas cannot provide adequate information required by the financial institution to make a lending decision. As a result, it is challenging for microfinance institutions to assign credit properly based on conventional policies. In this paper, we formulate the decision-making of microfinance into a rigorous optimization-based framework involving learning and control. We propose an algorithm to explore and learn the optimal policy to approve or reject applicants. We provide the conditions under which the algorithms are guaranteed to converge to an optimal one. The proposed algorithm can naturally deal with missing information and systematically tradeoff multiple objectives such as profit maximization, financial inclusion, social benefits, and economic development. Through extensive simulation of both real and synthetic microfinance datasets, we showed our proposed algorithm is superior to existing benchmarks. To the best of our knowledge, this paper is the first to make a connection between microfinance and control and use control-theoretic tools to optimize the policy with a provable guarantee.Preprint. Under review.