Raman spectrum is wealth of structural information, which can be used as molecular fingerprint to identify compounds. There are relations between the intensities of Raman peaks and the concentrations of compound, and they can be used to estimate relative concentrations of the compounds in mixture. However, it is still challenging to interpret the Raman spectrum of mixture. The non-negative lasso (NN-LASSO) has been used to solve this problem, but it failed to identify highly correlated compounds. The quadratic term of nonnegative elastic net (NN-EN) can ensure the stability of the fitted model. Therefore, a novel mixture analysis method was developed based on NN-EN in this study. It has been applied to analyze the simulated, liquid, powder, tablet, and quantitative mixture datasets. Results showed that NN-EN can identify the compounds in mixture with high accuracy and estimate their relative concentrations with small deviation. Furthermore, NN-EN was more stable than NN-LASSO when the spectra of some compounds are highly correlated. It is a promising approach for analyzing Raman spectra of mixtures.