Necessary model calculation simplifications, uncertainty in actual wind tunnel test, and data acquisition system error altogether lead to error between a set of actual experimental results and a set of theoretical design results; wind tunnel test flutter data can be utilized to feedback this error. In this study, a signal processing method was established to use the structural response signals from an aeroelastic model to classify flutter signals via deep learning algorithm. This novel flutter signal processing and classification method works by combining a convolutional neural network (CNN) with time-frequency analysis. Flutter characteristics are revealed in both time and frequency domains, which are harmonic or divergent in the time series; the flutter model energy is singular and significantly increases in the frequency view, so the features of the time-frequency diagram can be extracted from the dataset-trained CNN model. As the foundation of the subsequent deep learning algorithm, the datasets are placed into a collection of time-frequency diagrams calculated by short-time Fourier transform (STFT) and labeled with two artificial states, flutter or no flutter, depending on the source of the signal measured from a wind tunnel test on the aeroelastic model. After preprocessing, a cross-validation schedule is implemented to update (and optimize) CNN parameters though the trained dataset. The trained models were compared against test datasets to validate their reliability and robustness. Our results indicate that the accuracy rate of test datasets reaches 90%. The trained models can effectively and automatically distinguish whether or not there is flutter in the measured signals.