It is important but challenging for prospective health professionals to learn the visual distinction between potentially harmful and harmless skin lesions, such as malignant melanomas and benign nevi. Knowledge about factors related to diagnostic performance is sparse but a prerequisite for designing and evaluating evidence-based educational interventions. Hence, this study explored how the characteristics of 240 skin lesions, the number of classified lesions and the response times of 137 laypeople were related to performance in diagnosing pigmented skin cancer. Our results showed large differences between the lesions, as some were classified correctly by more than 90% and others by less than 10% of the participants. A t-test showed that for melanomas, the correct diagnosis was provided significantly more often than for nevi. Furthermore, we found a significant Pearson correlation between the number of solved tasks and performance in the first 50 diagnostic tasks. Finally, t-tests for investigating the response times revealed that compared to true decisions, participants spent longer on false-negative but not on false-positive decisions. These results provide novel knowledge about performance-related factors that can be useful when designing diagnostic tests and learning interventions for melanoma detection.