Paroxysmal atrial fibrillation (PAF) is the initial stage of atrial fibrillation, one of the most common arrhythmia types. PAF worsens with time and affects the patient's life quality negatively. In this study, we aimed to early diagnose PAF patients, so patients can start taking precautions before this disease gets worse. We used the Atrial Fibrillation Prediction Database, an open data from Physionet, and constructed our approach using convolutional neural networks. Heart rate variability (HRV) features are calculated from time-domain measures, frequency-domain measures using power spectral density estimations (fast Fourier transform, Lomb-Scargle, and Welch periodogram), timefrequency-domain measures using wavelet transform, and nonlinear Poincare plot measures. We also normalized these features using min-max normalization and z-score normalization methods. In addition, we also applied alternatively the heart rate normalization (HRN), which gave promising results in a few HRV-based research, before calculating these features. Thus, HRV data, HRN data, and HRV features extracted from six different combinations of these normalizations, in addition to no normalization cases, were applied to the convolutional neural networks classifier. We tuned the classifiers using 90% of samples and tested the classifiers' performances using 10% of data. The proposed approach resulted in 95.92% accuracy, 100% precision, 91.84% recall, and 95.74% f1-score in HRV with z-score feature normalization. When the heart rate normalization was also applied, the proposed approach reached 100% accuracy, 100% precision, 100% recall, and 100% f1-score in HRV with z-score feature normalization. The proposed method with heart rate normalization and z-score normalization methods resulted in better classification performance than similar studies in the literature. In addition, although deep learning models offer no use of separate feature extraction processes, this study reveals that using HRV-specific feature extraction techniques may improve the performance of deep learning algorithms in HRV-based studies. Comparing the existing studies, we concluded that our approach provides a much better tool to diagnose PAF patients.