Discussing population growth management in Nigeria is largely viewed as politically incorrect. Whilst this research reiterates the rancorous circle which poverty casts on Nigeria's all-round efforts in managing her Total Fertility Rate; inclusive of social, environmental, cultural, political and economic insecurities which strongly affect women negatively, it negates the nature of a so-called absolute right to have as many children as we want, and identifies the need for a workable population programme and policy framework which must be rights-centered, promote sustainable development and enhance individual growth. In extending submissions with a view to overcoming challenges posed to a rights-based approach to fertility control, this paper draws out existing barriers to any practicable population management framework that can apply to our nation without profound frustration in our system, consequent impracticality or ultimate abandonment of such policy or programme.