The nature, extent and quality of mathematics learning among young children in India cannot be adequately understood without looking at the larger context of education and the social background of the children. Society, including schools, characterized by large inequalities impacts mathematics learning. Beginning with a brief overview of (mathematics) education in India, in historical and sociological perspectives, an appraisal is presented of the need and nature of mathematics learning revealed by field studies in two communities in a deprived rural setting and a low-income urban setting, respectively. While the latter was economically active, the former was much poorer in work and education opportunities, though had richer cultural practices that involved engagement with mathematical riddles, puzzles, folklores and mnemonic tables. The paper discusses the enabling potential of the knowledge resources, including work-context knowledge, which exist in both the communities despite the prevalent deprivations due to disadvantaged conditions. Yet in both situations mathematics learning remains disconnected from formal school mathematics. Factors within SES that possibly have strong bearings on mathematics learning are highlighted which can scaffold stronger integration with curricular and pedagogic practices. Both the groups presented potentially rich contexts for drawing upon everyday mathematical knowledge that can inform effective mathematics learning, which has been inadequately explored in curriculum and instructional design thus far.