Since the mid-20th century, anxiety has been identified as one of the debilitating factors for the success of second language acquisition (SLA) when researchers focus on the role of affective filters, namely motivation and personality factors, in enhancing learners’ cognitive capacities. This study focuses on the speech anxiety of female students in English language classrooms at the tertiary level in Bangladesh. This qualitative research used semi-structured interview techniques to investigate the individual perception of second language learners towards their speaking skills performance in the classroom or social settings. The sample size was 20 female respondents selected from two private universities in Bangladesh through purposive sampling. In order to investigate the learners' anxiety and the related factors behind anxiety systematically, an interview profile has been made where 20 sub-questions are framed under the associated themes of research questions. The findings of this research show not only their views on learners' anxiety but also a wide range of sociocultural and pedagogical factors that demotivate them from speaking in English in the broader social context. It further focuses on the role of the education system of Bangladesh in facilitating the teaching and learning of English speaking at the tertiary level properly. Finally, this study provides insight into the field of SLA showing some pathways to mitigate the speech anxiety of female learners at tertiary levels.