Deep learning-based approaches have become popular for automatically detecting defects in electroluminescence images of solar cells. However, deep learning methods are those that require the most training data among machine learning approaches. Thus, the data available to train such models is currently a bottleneck for their performances due to expensive and possibly inaccurate labeling. To address this problem, we propose to use a model comprising a standard deep learning classifier to which we add conformal prediction. The model calculates a degree of confidence on new predictions and can send low-confidence predictions for human expert labeling in an uncertainty-aware active learning loop. In tests with a limited-size data set, using the conformal model to select and classify high-confidence samples yields significantly higher performance compared to the standard deep learning classifier, as the F1 score increases from 0.44 to 0.62 while only leaving out 9.4% of predictions as low-confidence that need human assessment for validation and model update, demonstrating the effectiveness of the framework.