Eighty pianists each listened to 21 trials of solo piano music. Trials consisted of two different performances of the same excerpt, and the same music was played on all trials for any given subject. Over the 21 trials, seven different interpretations were presented in all possible pair-wise combinations. Forty subjects listened to a slow excerpt from Liszt's Totentanz, and the other forty listened to a fast excerpt from the same piece. The subjects' task was to select which of the two performances on each trial, if either, they preferred. Subjects were assigned randomly to one of four conditions: preference only, preference plus use of the musical score, preference plus use of rating scales, and preference plus use of both the musical score and rating scales ( n = 10 per treatment per excerpt). Results indicated that the use of the musical score and of rating scales both separately and in combination with each other did not improve consistency as compared to their nonuse. Subjects who responded to rating scales, however, were more consistent when they did not also use the musical score than when they did use it. Subjects were less consistent for the slow excerpt than they were for the fast excerpt, and consistency was unaffected by their piano experience. Presentation order within trials (first versus second excerpt) did not affect ratings; however, ratings on Trials 12-21 were slightly and significantly higher than ratings on Trials 1-10. Also, preference for a performance was not affected by preference for the performance immediately preceding it.