This paper examines differences in the user feedback of scholars with varied experience with digital tools. As part of a usability study of a historical digital collection, our team conducted semi-structured interviews with scholars with varying backgrounds. We categorized the sample into two groups, one with significant experience and one with little experience in using digital technology. Qualitative analysis of the interview data showed that users generally provided similar feedback. However, there were instances in which those with significant experience provided more design suggestions, and those with less experience expressed confusion and provided more feedback on website content. Drawing upon our findings, we provide recommendations for the usability evaluation of historical digital collections.