Upon arrival to a host community, refugees and asylum seekers face immense challenges to rebuild their social capital that is critical in the process of their resettlement. Developing a holistic understanding of these challenges can provide significant opportunities to inform designers and services providers working with this demography. We adopt the lens of social capital coupled with an inquiry into the heterogeneity of refugees and asylum seekers to gain a holistic understanding of various challenges that they with. We accordingly present a conceptual framework that has been iteratively built based on our four years of engagement with refugee communities. The framework highlights three important aspects: cultural backdrops, displacement-related stressors, and social resources in the host community. We offer several implications for technology design, policies, and the theory of social capital that can support members from these communities in their resettlement.