In Nigeria, the definition of small and medium enterprises (SMEs) varies between government agencies and sectors of the economy. These definitions have an impact on the survival of SMEs because of the associated implications on the formulation as well as the implementation of policies. This paper seeks to develop a framework towards developing a holistic definition of SMEs in the Nigerian Context and address some of the challenges arising from their external business environment. This alternative approach is developed by taking into consideration not only the nature and needs of SMEs in Nigerian through primary research, but also by examining some of the widely used definitions and key challenges available in the literature. This study argues that SMEs in Nigeria should be redefined using a holistic definition that will be beneficial to SMEs long terms sustainability and competitive advantage in the country.