In order to solve the problem of lack of multimodal emotional database, a computer speech recognition technology and graphic form design research were proposed. Using the hidden Markov model (HMM) to recognize speech emotion is mainly to provide emotion type for the subsequent expression fusion. When the emotion category in the speech is obtained, the facial animation parameter (FAP) corresponding to the speech emotion can be combined with the lip movement FAP based on the Moving Pictures Experts Group (MPEG-4) face animation standard to obtain a comprehensive FAP. The results show that the recognition effect of speech emotion recognition in the method used in this paper is relatively better than that obtained by other literature methods, and the average recognition rate reaches 70.24%, which is higher than the other three methods. It is verified that this method can present a better recognition effect.