Rapid and accurate electrocardiogram (ECG) signal classification is crucial in high-stakes healthcare settings. However, existing computational models often struggle to balance high performance with computational efficiency. This study introduces an innovative computational framework that combines transfer learning with traditional machine learning to optimize ECG classification. We use a pre-trained Stacked Convolutional Neural Network (SCNN) to generate high-dimensional feature embeddings, which are then evaluated by an array of machine learning classifiers. Our models demonstrate exceptional performance, particularly when utilizing embeddings from SCNNs trained on diverse datasets. This underscores the importance of data diversity in improving classifier discrimination. Notably, Multilayer Perceptrons (MLPs) stand out for their ability to balance computational efficiency with strong performance, achieving test F1-scores of 0.94 and 1.00 in multi-class and binary tasks on the CinC2017 dataset, and 0.85 and 0.99 on the CPSC2018 dataset. Our approach consistently outperforms existing methods, setting new benchmarks in ECG classification. The synergy between deep learning-based feature extraction and traditional machine learning through transfer learning offers a robust, efficient, and adaptable strategy for ECG classification, addressing a critical research gap and laying the groundwork for future advancements in this crucial healthcare field.