Compared with the large number of studies on reading of children with hearing loss (HL) in alphabetic languages, there are only a very limited number of studies on reading of Chinese-speaking children with HL. It remains unclear how phonological, orthographic, and semantic skills contribute to reading fluency of Chinese school-age children with HL. The present study explored this issue by examining the performances of children with HL on reading fluency and three linguistic skills compared with matched controls with normal hearing (NH). Specifically, twenty-eight children with HL and 28 chronological-age-matched children with NH were tested on word/sentence reading fluency (WRF/SRF), phonological awareness (PA) which was composed of onset/vowel/lexical tone awareness, orthographic knowledge (OK), and semantic competence (SC) which comprised animal word identification, pseudo-homophone detection, and word segmentation. Results showed that children with HL lagged behind their peers with NH in WRF/SRF and most of the phonological, orthographic, and semantic subskills except onset awareness and pseudo-homophone detection. Furthermore, the significant contributors to WRF differed between the two groups with PA being the significant contributor in the children with NH while OK being the significant contributor in the children with HL. However, the significant contributor to SRF did not differ between the two groups with SC being the only significant contributor. These results revealed not only between-group differences but also similarities in the relative contributions of PA, OK, and SC to reading fluency at both word and sentence levels, which has practical implications for developing better training programs to improve reading for children with HL.