Recently, cardiac arrhythmia recognition from electrocardiography (ECG) with deep learning approaches is becoming popular in clinical diagnosis systems due to its good prognosis findings, where expert data preprocessing and feature engineering are not usually required. But a lightweight and effective deep model is highly demanded to face the challenges of deploying the model in real-life applications and diagnosis accurately. In this work, two effective and lightweight deep learning models named Deep-SR and Deep-NSR are proposed to recognize ECG beats, which are based on two-dimensional convolution neural networks (2D CNNs) while using different structural regularizations. First, 97720 ECG beats extracted from all records of a benchmark MIT-BIH arrhythmia dataset have been transformed into 2D RGB (red, green, and blue) images that act as the inputs to the proposed 2D CNN models. Then, the optimization of the proposed models is performed through the proper initialization of model layers, on-the-fly augmentation, regularization techniques, Adam optimizer, and weighted random sampler. Finally, the performance of the proposed models is evaluated by a stratified 5-fold cross-validation strategy along with callback features. The obtained overall accuracy of recognizing normal beat and three arrhythmias (V-ventricular ectopic, S-supraventricular ectopic, and F-fusion) based on the Association for the Advancement of Medical Instrumentation (AAMI) is 99.93%, and 99.96% for the proposed Deep-SR model and Deep-NSR model, which demonstrate that the effectiveness of the proposed models has surpassed the state-of-the-art models and also expresses the higher model generalization. The received results with model size suggest that the proposed CNN models especially Deep-NSR could be more useful in wearable devices such as medical vests, bracelets for long-term monitoring of cardiac conditions, and in telemedicine to accurate diagnose the arrhythmia from ECG automatically. As a result, medical costs of patients and work pressure on physicians in medicals and clinics would be reduced effectively.