The significant difference between recycled aggregate and natural aggregate is the content of the attached mortar layer. With the increase of the replacement rate of recycled aggregate, the shrinkage and creep of recycled aggregate concrete is significantly increased. In this paper, 180-day shrinkage and creep tests of recycled aggregate concrete with different water–cement ratios were designed in order to analyze the effect of the substitution rate and water–cement ratio on shrinkage and creep properties. The results show that the shrinkage strain of recycled aggregate concrete with a substitution rate of 50% and 100% at 180 days is 26% and 48% higher than that of ordinary concrete, respectively, and the growth of group II is 22% and 47%, respectively. When the load was 180 days old, the creep coefficient of recycled aggregate concrete with a substitution rate of 50% and 100% in group I increased by 19.6% and 39.6%, respectively compared with ordinary concrete, and group II increased by 23.6% and 44.3%, respectively. Based on the difference of adhering mortar content, the creeping increase coefficient and shrinkage increase coefficient of the attached mortar were proposed, and a shrinkage and creep model of recycled aggregate concrete was established. When compared with the experimental results, the model calculation results met the accuracy requirements.