We consider two different methods of calculating the relevant average for the non-equilibrium partition identity (NPI), i.e. k exp ½2 V t tl, which result in two different values. At best only one of these will accurately correspond to what is observed. In order to better understand the two outcomes we carry out a detailed error analysis. This analysis is difficult due to the importance of extremely rare events in forming the average, resulting in the necessity to go beyond linear approximations for the error estimates. We begin by analysing the error in the fluctuation relation, and build upon this to estimate the errors in the NPI average. At short durations the full ensemble average always gives the observed average (i.e. the NPI holds). However, at very long durations, given a fixed amount of sampling, the observed average is predicted by treating the probability distribution as a Dirac-delta function. At intermediate times, neither corresponds to the observed average. This has profound implications for non-equilibrium work relations, as first introduced by Jarzynski.