In everyday conversation, we often use indirect replies to save face of our interlocutor (e.g., “Your paper does have room for improvement”). Six experiments were conducted to examine the role of verbal and nonverbal behaviors in the production and comprehension of indirect replies. In Experiments 1a and 1b, participants engaged in question-answer exchanges designed to elicit four types of replies (i.e., direct, indirect, lie, and neutral). Results showed that uncertainty terms, discourse markers and head tilt were most uniquely associated with the production of indirect replies. In Experiments 2a, 2b, 3a and 3b, participants categorized the types of replies in video clips of real participants in Experiments 1a and 1b. Results showed that nonverbal behaviors enhanced the performance and boosted the confidence in the identification of indirect replies. Furthermore, uncertainty terms, discourse markers and head tilt were also the most reliable cues for identifying indirect replies. Finally, the extent to which people relied on verbal and nonverbal cues to identify an indirect reply was context dependent. The more informative the verbal/nonverbal information was, the fewer nonverbal/verbal cues that contributed to the identification of indirect replies. Our results demonstrated that people integrate verbal and nonverbal information to enhance their understanding of the intended meaning in indirect replies. Findings from the current research provide an initial step toward developing a comprehensive and unified model of the production and comprehension of indirect replies, which takes both verbal and nonverbal behaviors into account.