The COVID-19 pandemic has profoundly affected many world regions’ coastal social-ecological systems (SESs). Its extensive consequences have exposed flaws in numerous facets of society, including small-scale coastal fisheries in developing countries. To this extent, by focusing on two coastal districts in Bangladesh, namely Chittagong and Cox’s Bazar, we investigated how the lockdown during COVID-19 impacted small-scale coastal fishers in Bangladesh and which immediate measures are required to develop and implement insights, on the role of the scale of governance attributes, in facilitating or impeding the resilience of small-scale fisheries (SSFs). We analyzed both qualitative and quantitative data obtained through semi-structured, in-depth individual interviews (n = 120). Data were further validated using two focus group discussions in the study areas. The impact of the pandemic on the fishers’ livelihood included halting all kinds of fishing activities; limited time or area for fishing; livelihood relocation or alternative work; low fish price; fewer fish buyers, causing difficulty in selling; and travel or free-movement restrictions. Additionally, the study discovered several coping skills and found that the most prevalent coping strategy against the effect of the COVID-19 pandemic was to take out loans (48%) from different organizations and NGOs and borrow money from relatives, neighbors, friends, or boat owners. Finally, the current research analysis identified possible recommendations to enhance the resilience of coastal fishers during COVID-19, emphasizing arrangements that should be made to provide alternative livelihood opportunities for coastal fishermen via need-based training, technical and vocational education and training, and microcredit to keep them afloat and earning during the pandemic, not relying only on fishing.