We study the emergence of reference points in a bilateral, infinite horizon, alternating offers bargaining game. Players' preferences exhibit reference dependence, and their current offers have the potential to influence each other's future reference points. However, this influence is limited in that it expires in a finite number of periods. We first construct a subgame perfect equilibrium that involves an immediate agreement and study its properties. Later, we also show the existence of an equilibrium where agreement is reached with delay. We show that expiration lengths and initial reference points play a crucial role for the existence of this equilibrium. For instance, we show that equilibrium with a delayed agreement does not exist when the initial reference point is (0, 0). Finally, we provide comparative static analyses on model parameters, compare two variations of our model, and compare our findings with those of the closest paper to ours, Driesen et al. (Math Soc Sci 64:103-118, 2012).