While discourse competence (DC) has been identified as the core competence in English as a foreign language (EFL) learning and communication, only a few empirical studies have focused on DC in speaking assessments. This study investigates the discourse features (i.e., cohesion and coherence) of Taiwanese candidates’ performances in an argumentative exposition task in the General English Proficiency Test (GEPT) by analyzing the construction of cohesion and coherence across four band scores (i.e., Bands 2, 3, 4, and 5). Cohesion analysis showed that all the three types of cohesion devices (i.e., grammatical, logical, and lexical) develop linearly with increasing band score levels. The higher-score test-takers (Bands 4 and 5) adopted grammatically and semantically more appropriate references to establish an accurate relation with the referents. They also tended to use a higher frequency of overall conjunctions (particularly additive) and repetition to promote logical and lexical cohesion. Coherence analysis found that high-score candidates were more equipped to produce a response that complies with the expected generic structure of argumentative tasks. While more complex theme-rheme patterns appeared more frequently in the discourse of higher-score candidates, theme reiteration patterns were only observed in Band 2. These results illustrate how Taiwanese EFL learners demonstrate DC with cohesive and coherence strategies in speaking test performance across band scores. Pedagogical implications for language teaching and learning are then discussed.