<b><i>Introduction:</i></b> Several factors are known to influence speech perception in cochlear implant (CI) users. To date, the underlying mechanisms have not yet been fully clarified. Although many CI users achieve a high level of speech perception, a small percentage of patients does not or only slightly benefit from the CI (poor performer, PP). In a previous study, PP showed significantly poorer results on nonauditory-based cognitive and linguistic tests than CI users with a very high level of speech understanding (star performer, SP). We now investigate if PP also differs from the CI user with an average performance (average performer, AP) in cognitive and linguistic performance. <b><i>Methods:</i></b> Seventeen adult postlingually deafened CI users with speech perception scores in quiet of 55 (9.32) % (AP) on the German Freiburg monosyllabic speech test at 65 dB underwent neurocognitive (attention, working memory, short- and long-term memory, verbal fluency, inhibition) and linguistic testing (word retrieval, lexical decision, phonological input lexicon). The results were compared to the performance of 15 PP (speech perception score of 15 [11.80] %) and 19 SP (speech perception score of 80 [4.85] %). For statistical analysis, U-Test and discrimination analysis have been done. <b><i>Results:</i></b> Significant differences between PP and AP were observed on linguistic tests, in Rapid Automatized Naming (RAN: <i>p</i> = 0.0026), lexical decision (LexDec: <i>p</i> = 0.026), phonological input lexicon (LEMO: <i>p</i> = 0.0085), and understanding of incomplete words (TRT: <i>p</i> = 0.0024). AP also had significantly better neurocognitive results than PP in the domains of attention (M3: <i>p</i> = 0.009) and working memory (OSPAN: <i>p</i> = 0.041; RST: <i>p</i> = 0.015) but not in delayed recall (delayed recall: <i>p</i> = 0.22), verbal fluency (verbal fluency: <i>p</i> = 0.084), and inhibition (Flanker: <i>p</i> = 0.35). In contrast, no differences were found hereby between AP and SP. Based on the TRT and the RAN, AP and PP could be separated in 100%. <b><i>Discussion:</i></b> The results indicate that PP constitute a distinct entity of CI users that differs even in nonauditory abilities from CI users with an average speech perception, especially with regard to rapid word retrieval either due to reduced phonological abilities or limited storage. Further studies should investigate if improved word retrieval by increased phonological and semantic training results in better speech perception in these CI users.