In recent years, a wide variety of Machine Learning (ML) algorithms, including Deep Learning (DL) methods, have been proposed for electrocardiogram (ECG) beat classification. However, accurately discerning ECG beat types faces challenges due to noise interference and inherent imbalances among different classes. Moreover, understanding mathematical models enclosed by black-box learning systems is an open issue today. Our study employed a manifold learning algorithm capable of mapping highdimensional data into a latent space to conduct a comprehensive analysis within a neural network learning framework. This approach involved the following studies: (1) Examining the intermediate high-dimensional latent space in simple architectures by studying its projection into a visualizable latent space; (2) Exploring the influence of class imbalance on the configuration of the latent space; (3) Evaluating and analyzing the compensatory effects of employing diverse DL architectures, such as modified autoencoders and Generative Adversarial Networks (GANs), specifically in generating data augmentation through synthetic beats. The experimental results demonstrated the effectiveness of our methodology in mitigating noise and addressing inter-class imbalance, notably enhancing the diagnostic Area Under the ROC Curve (AUC) in ECG signal analysis. Implementation of GAN data augmentation techniques resulted in a 2% improvement, elevating the AUC from 0, 9332 to 0, 9520 in the biclass dataset. Similarly, the AUC values increased by 2%, from 0, 9020 to 0, 9223, for the multiclass dataset. These findings highlight the impact of appropriate data augmentation techniques on AUC improvement. Furthermore, visualizing latent spaces during beat classifier design significantly contributes to the development of solid and principled multiclass beat-discriminating systems.