Previous studies have established that male and female speakers differ in their production of foreign language (FL) vowels due to biological differences in the vocal tract. These differences present a challenge when subjects recruited for an instrumental analysis of speech sounds include speakers of both genders. Most previous studies, therefore, focus on only one gender to avoid the problem of biological differences between males and females. However, different gender groups may perform differently in speaking tasks as their language experience may be influenced by other social factors. Therefore, the aim of this study is to assess the differences between Libyan male and female speakers in the production of English vowels, by examining the data with and without the use of normalization to determine the most difficult English vowels for Libyan male and female speakers. To achieve this goal, seventy Libyan participants (35 male and 35 female) were recruited to perform a speech production task of eleven English vowels embedded in carrier sentences. The data were collected and analyzed using PRAAT to extract the F1 and F2 formant frequencies for each vowel and to calculate the Euclidean distance (ED) of the English vowels produced by using the centroid as the reference point. With the unnormalized data, more overlaps were found for the male speakers when compared to the female speakers as the vowel space were more centred for the males compared to the females. With the unnormalized data, different pairs of vowel contrasts were identified as potentially problematic. However, with the normalized data, better categorical distinction is found in general. Only three vowels emerged as potentially problematic as the other pairs could potentially be disambiguated using vowel duration as a cue. The study demonstrated the utility of the Lobanov method for normalizing speech data to reduce variability associated to gender differences. The efficacy of the Lobanov method of normalization remains to be tested further to verify if the predictions made about difficult vowel pairs for Libyan EFL learners are indeed attested in their discrimination of these vowel pairs in future research.