Entrepreneurship is major source of jobs creation, economic growth, and innovation in most of economies. It entails identification of opportunity, and a series of steps taken to commercialize the identified opportunity in profitable manner. Along the way, entrepreneur encounters many decision problems pertaining to strategy and operation, ranging from deciding on the business opportunity to go-to-market plan. Decision-making in entrepreneurial setting becomes difficult and challenging because of many unknowns requiring knowledge of contextual intelligence and changing business environment, further accentuated in emerging economies because of regulatory uncertainty and change. Right decision-making is key to profitable exit or early disengagement, resulting in reduced sunk cost. The paper proposes a framework for decision-making in entrepreneurial setting based on experimentation. We posit that entrepreneurial decision-making can be modelled in the settings of contextual bandits. In the process, we detail out aspects of decision problems, entrepreneurial ecosystem, and experiments, which impacts decision-making processes and associated cost. We develop propositions pertaining to association between decision problems, ecosystem, and entrepreneurial growth. Based on that, we draw implications for entrepreneurs and policymakers.