Perceived trustworthiness based on facial appearance plays an important role in interpersonal trust and cooperative behavior. Interpersonal trust behaviors involve both trustors and trustees. However, there is no clear conclusion on how the age of the two individuals affects interpersonal trust behaviors. Therefore, this study used the trust game task to explore the differences in trust behaviors between two different age groups in response to faces of different ages and analyzed whether such differences were apparent in the face processing stage. The behavioral results showed that only younger adults invested more money with older partners than younger ones; that is, younger adults trusted older faces more. The event-related potential (ERP) analyses showed that in the early stage of face processing, younger faces elicited more negative N170 than older faces; at the same time, older faces elicited more positive VPP than younger faces, and younger adults had more positive VPP than older adults. In the middle and late stages of face processing, younger faces elicited more negative FRN than older faces in younger adults but not in older adults. In addition, older faces elicited more positive LPP than younger faces in older adults but not in younger adults. The neural analyses suggested that age-related differences in facial trustworthiness judgments might occur in the later stages of face processing. Combining the behavioral and neural results, we found a dissociation between trustworthiness perceptions and trust behaviors in both younger and older adults, which may provide insight into how to prevent older adults from being deceived.