Adhesively bonded carbon fiber reinforced polymer (CFRP)/Al joints have been widely used in engineering field. However, the bonding strength still needs to be improved. In this study, ultrasonic vibration was applied during the adhesive bonding to promote the micromechanical anchoring at the interface of CFRP/Al joints. Ultrasonic vibration was exerted on the CFRP laminate to transmit the vibration to the adhesive bonding area once the joint was assembled. The strength and the strength consistency of the joints were increased by 45% and 50%, respectively. Scanning electron microscope (SEM) and energy dispersive spectrometer (EDS) results showed that the ultrasonic vibration promoted the filling of the adhesive into the microstructures of the adherends and achieved a more compact bonding. Viscosity and contact angle of the adhesive were measured, and both of them were decreased greatly under the ultrasonic vibration, indicating that the ultrasonic vibration can increase the fluidity of the adhesive and force the wetting between the materials. Ultrasonic capillary experiment was then conducted. The adhesive in the capillary tube was prompted to rise obviously, and the meniscus was inverted with ultrasonic vibration, showing that the ultrasonic vibration produced a driving effect on filling of the adhesive in the microstructures. Therefore, the joints exhibited improved bonding strength owing to the enhanced micromechanical anchoring and physical adsorption at the bonding interface. This study is of great significance for understanding and optimization of the ultrasonic vibration strengthened adhesive bonding.