Sound symbolism refers to the non-arbitrary relationship between phonemes and specific perceptual attributes. Few studies have focused on the sound symbolic associations between Mandarin phonemes and multiple perceptual dimensions, including social attitudes. The main purpose of the current study is to identify the acoustic cues crucial to perceptual judgment on the visual, tactile, and interpersonal dimensions based on the Mandarin rimes. The study found that in addition to the first and second formants, formant transitions and nasal codas were crucial in characterizing sound symbolic associations. Machine learning models also showed that critical acoustic parameters could successfully classify the different perceived attributes of rimes. The study further examined the mechanisms underlying forming a sound symbolic association. Through mediation analyses, the light/heavy dimension proved to have a suppression effect on the politeness and friendliness perception of Mandarin compound rimes (diphthongs and diphthongs with nasal codas /n/ or /ŋ/). This finding suggested that the two potential mechanisms of sound symbolism (i.e., language pattern account and shared-property account) could coexist and interact with each other.