Three types of self-piercing riveting (SPR) joints, i.e., steel/aluminum, carbon fiber reinforced polymer (CFRP)/ aluminum, and aluminum/aluminum, were constructed using three different top sheet materials with the same aluminum alloy as the bottom sheet. The effects of the top sheet material on the joint quality and mechanical behavior were evaluated. The top sheet materials’ characteristics dominate the rivet piercing process and the consequent interlock distance. The high-strength steel top sheet requires a comparatively higher rivet setting force and induces early flaring of the rivet tail, resulting in a larger interlock distance. Though the CFRP needs the highest rivet setting force to penetrate the rivet through the CFRP fibers, the CFRP/aluminum joint exhibits the smallest interlock distance because of the SPR process-induced damages to the CFRP and subsequently less flaring of the rivet tail. In strength tests, the damaged CFRP sheet resulted in rivet head pullout of the CFRP/aluminum joints, which exhibited the lowest lap-shear and cross-tension strengths. In contrast, the steel/aluminum joints demonstrated the highest strengths because of their comparatively larger interlock distances. In addition to the experimental analysis, simulations revealed the rivet penetration and flaring mechanisms with various top sheet materials, and their respective joint quality and strengths.