The aim of the study was to investigate to what extent the vocabulary and reading components of the Test for English Majors Band 4 (TEM-4) do indeed measure the constructs they are supposed to measure. The concurrent validity of the reading and vocabulary components of the test was studied by correlating results on the TEM-4 with widely used tests of vocabulary and reading comprehension. A total of 60 English Major students, in their second year at a university in North China, were tested using the bilingual Mandarin-English version of the Vocabulary Size Test (Nation & Beglar, 2007), the Vocabulary Knowledge Scale (Brown, 2008), which assesses depth of vocabulary knowledge and the TEM-4. 30 out of these 60 students were also individually tested using the York Assessment of reading comprehension Secondary (Snowling et al., 2009). The results show that the reading component of the TEM-4 did not correlate with the YARC. Instead there were modest correlations between the TEM-4 reading component, on the one hand, and the VST and the VKS on the other hand. It is therefore likely that the reading component of the TEM-4 taps into vocabulary knowledge rather than reading. Regression analyses also confirmed these findings. We conclude that the reading component of the TEM-4 underrepresents the construct of reading, and that the theoretical underpinning and the ways in which reading is assessed in this test will need to be reconsidered.