This paper investigates the effects of expanded perlite aggregate (EPA) on the properties of mortar and concrete. A volume of 10, 20, and 30% EPA were used instead of 0–2 mm fine aggregate. Water/binder ratio of 0.30 was constant for all the mortar and concrete mixtures. The compressive strength, ultrasonic pulse velocity (UPV), and linear autogenous shrinkage tests were performed on both mortar and concrete mixtures. Furthermore, the heat of hydration test was only made for mortar mixtures and restrained shrinkage test was only employed to measure cracking time and strain for concrete mixtures. For both mortar and concrete mixtures, the experiments show that the compressive strength, UPV, and linear autogenous shrinkage were generally decreased with increasing the amount of EPA according to control sample, while the heat of hydration is generally increased. However, the reduction ratios in the compressive strength of mortar and concrete samples gradually decreased with increasing the curing time. In addition, the increases in the heat of hydration for EPA10, EPA20, and EPA30 were 5, 7, and 8%, compared with the control.