People hear differently and have different aesthetic criteria. Due to a number of these reasons, individuals may prefer different processing parameters for both speech and music. For example, some individuals may prefer more low frequency emphasis and longer reverberation time while listening to music. Alternatively, certain individuals with hearing loss may prefer higher compression ratios at high frequencies. To perform this “personalization,” a number of psychometric procedures can be used. They vary in terms of accuracy, complexity, testing time, etc. For consumer devices, such as headsets, personal amplifiers, personal music players, etc., we have developed a new methodology, called EarPrint™, which is fast, intuitive, and requires minimum instructions. The procedure is based on reducing a multidimensional parametric space into a 2-D surface representation where the user perceives a consistent quality increment as he/she moves his/her finger along a straight line. In this paper, we evaluate how valid and reliable the EarPrint™ results are when compared to more rigorous psychometric procedures (e.g., category scaling or magnitude estimates.) Fifteen subjects were tested and the results were analyzed in terms of validity, dependence on listening material and reliability.