The mental stress faced by many people in modern society is a factor that causes various chronic diseases, such as depression, cancer, and cardiovascular disease, according to stress accumulation. Therefore, it is very important to regularly manage and monitor a person’s stress. In this study, we propose an ensemble algorithm that can accurately determine mental stress states using a modified convolutional neural network (CNN)- long short-term memory (LSTM) architecture. When a person is exposed to stress, a displacement occurs in the electrocardiogram (ECG) signal. It is possible to classify stress signals by analyzing ECG signals and extracting specific parameters. To maximize the performance of the proposed stress classification algorithm, fast Fourier transform (FFT) and spectrograms were applied to preprocess ECG signals and produce signals in both the time and frequency domains to aid the training process. As the performance evaluation benchmarks of the stress classification model, confusion matrices, receiver operating characteristic (ROC) curves, and precision-recall (PR) curves were used, and the accuracy achieved by the proposed model was 98.3%, which is an improvement of 14.7% compared to previous research results. Therefore, our model can help manage the mental health of people exposed to stress. In addition, if combined with various biosignals such as electromyogram (EMG) and photoplethysmography (PPG), it may have the potential for development in various healthcare systems, such as home training, sleep state analysis, and cardiovascular monitoring.