Photo analysis offers a simple, noninvasive approach to characterizing and quantifying skin lesions in cetaceans; however, this process involves methodological considerations that have often gone unaddressed or have varied in approach among investigators. Subjectivity associated with classifying skin lesion types of unknown etiology and quantifying measures of skin lesion prevalence and extent from photo data raises questions about observer bias and agreement (i.e., interrater reliability), which are often ignored. The purpose of the present study was to improve upon data quality control and assessment practices when studying skin lesions using only photo data. Specifically, we tested interrater reliability of a skin lesion classification system, compared methods of quantifying skin lesion extent, and determined the validity of the dorsal fin as a proxy for skin lesions on the entire body. Acceptable levels of interrater reliability were achieved for only 7 of 17 defined lesion types, but reliability was high for the two tested measures of lesion extent. Skin lesion extent measured from the dorsal fin alone was not a decent proxy for the whole visible surface; disparities between measures were as high as 43%. We discuss the potential pitfalls discovered and provide recommendations for others attempting similar approaches.