Timbre is one of the psychophysical cues that has a great impact on affect perception, although, it has not been the subject of much cross-cultural research. Our aim is to investigate the influence of timbre on the perception of affect conveyed by Western and Chinese classical music using a cross-cultural approach. Four listener groups (Western musicians, Western nonmusicians, Chinese musicians, and Chinese nonmusicians; 40 per group) were presented with 48 musical excerpts, which included two musical excerpts (one piece of Chinese and one piece of Western classical music) per affect quadrant from the valence-arousal space, representing angry, happy, peaceful, and sad emotions and played with six different instruments (erhu, dizi, pipa, violin, flute, and guitar). Participants reported ratings of valence, tension arousal, energy arousal, preference, and familiarity on continuous scales ranging from 1 to 9. ANOVA reveals that participants’ cultural backgrounds have a greater impact on affect perception than their musical backgrounds, and musicians more clearly distinguish between a perceived measure (valence) and a felt measure (preference) than do nonmusicians. We applied linear partial least squares regression to explore the relation between affect perception and acoustic features. The results show that the important acoustic features for valence and energy arousal are similar, which are related mostly to spectral variation, the shape of the temporal envelope, and the dynamic range. The important acoustic features for tension arousal describe the shape of the spectral envelope, noisiness, and the shape of the temporal envelope. The explanation for the similarity of perceived affect ratings between instruments is the similar acoustic features that were caused by the physical characteristics of specific instruments and performing techniques.

In a cross-cultural context, exploring musical elements' cultural specificity and universality that affect various types of music is conducive to personalised emotion recognition. In this study, high-level musical elements are introduced to explore their influence on emotional perception. By comparing music emotion recognition (MER) models of varied cultural music, musical elements with cultural universality and cultural specificity are further determined. Participants rated valence, tension arousal, and energy arousal on labelled nine-point analogical-categorical scales for four types of classical music: Chinese ensemble, Chinese solo, Western ensemble, and Western solo. Fifteen musical elements in five categories-timbre, rhythm, articulation, dynamics, and register were annotated through manual evaluation or the automatic algorithm. The relationship between music emotion and musical elements was analysed through partial least squares regression. Results showed that tempo, rhythm complexity, and articulation are culturally universal; musical elements related to timbre, register, and dynamics features are culturally specific. By increasing tempo, rhythm complexity, staccato, perception of valence, tension arousal, and energy arousal can be effectively improved. Based on the Partial least squares regression (PLSR) model's results for the datasets, the combination of manual and automatic annotation for musical elements can improve the MER system's performance.