Atrial fibrillation (AF) is the most common heart rhythm disturbance in clinical practice. It often starts with asymptomatic and very short episodes, which are extremely difficult to detect without long-term monitoring of the patient’s electrocardiogram (ECG). Although recent portable and wearable devices may become very useful in this context, they often record ECG signals strongly corrupted with noise and artifacts. This impairs automatized ulterior analyses that could only be conducted reliably through a previous stage of automatic identification of high-quality ECG intervals. So far, a variety of techniques for ECG quality assessment have been proposed, but poor performances have been reported on recordings from patients with AF. This work introduces a novel deep learning-based algorithm to robustly identify high-quality ECG segments within the challenging environment of single-lead recordings alternating sinus rhythm, AF episodes and other rhythms. The method is based on the high learning capability of a convolutional neural network, which has been trained with 2-D images obtained when turning ECG signals into wavelet scalograms. For its validation, almost 100,000 ECG segments from three different databases have been analyzed during 500 learning-testing iterations, thus involving more than 320,000 ECGs analyzed in total. The obtained results have revealed a discriminant ability to detect high-quality and discard low-quality ECG excerpts of about 93%, only misclassifying around 5% of clean AF segments as noisy ones. In addition, the method has also been able to deal with raw ECG recordings, without requiring signal preprocessing or feature extraction as previous stages. Consequently, it is particularly suitable for portable and wearable devices embedding, facilitating early detection of AF as well as other automatized diagnostic facilities by reliably providing high-quality ECG excerpts to further processing stages.