English-only instruction (EOI) is widely adopted to enhance young learners’ comprehensive English capabilities in China, while little did we know about the difficulties teachers have. This study aimed to determine the difficulties teachers perceive during their English teaching to young learners with EOI in China. This case study’s data consisted of a survey, in-class observation, and in-depth interviews with two teachers in an English institute in Shenzhen. Results from thematic analysis of the data show that teachers face three types of difficulties, including teacher-related, student-related, and institutional. Teachers’ use of Chinese in lessons, lack of related pedagogic skills and experience, personalities, and self-recognition lead to teacher-related difficulties. Young learners’ short attention spans and safety issues are concluded as student-related difficulties. Institutional difficulties are the headmaster’s management style and appointed textbooks.