This paper presents a new approach for Malaysian speaker and accent recognition using wavelet feature extraction method, namely Wavelet Packet Transform (WPT), Discrete Wavelet Packet Transform (DWPT) and Dual Tree Complex Wavelet Packet Transform (DT-CWPT). Since Singular Value Decomposition (SVD) was based on factorization and summarization technique which reduces a rectangular matric, it is applied on those features to evaluate the performance for speaker and accent recognition. The features are derived from wavelets and SVD classified with three different classifiers namely k-Nearest Neighbors (k-NN), Support Vector Machine (SVM) and Extreme Learning Machine (ELM). In this work, English digits (0-9) and Malay words database uttered from 75 undergraduate students of Universiti Malaysia Perlis (UniMAP) which are Malays, Chinese and Indian. The Malay words had a combination of consonants and vowels in monosyllable and bi-syllable structure. The accuracy of file-based analysis achieved were above 81% while for frame-based analysis, 93.87% and above were obtained using three different classifiers (k-NN, SVM and ELM) for speaker and accent recognition. Through the experiments, it is observed that accent recognition achieved high recognition rate of 100% for both framed-based analysis and file-based analysis using SVM. The experimental results show the proposed features using SVD achieved high accuracy of 100% using SVM through English digits and Malay words in accent recognition. This indicated that feature extraction using wavelets (WPT, DWPT and DT-CWPT) with SVD can achieve a good performance for both English digits and Malay words.