Buffer-aided relaying has recently attracted a lot of attention due to the improvement in the system throughput. However, a side effect usually deemed is that buffering at relay nodes results in the increase of packet delays. In this paper, we study the effect of buffering at relays on the end-to-end delay of users' data, from the time they arrive at the source until delivery to the destination. We use simple discussions to provide an insight on the overall waiting time of the packets in the system, taking into account the queue dynamics both in the source and relay. We analyze the endto-end delay in the relay networks with Bernoulli data arrivals and channel conditions and prove that the data packets experience lower average end-to-end delay in the buffer-aided relaying system compared with the conventional one. Moreover, using intuitive generalizations, we conclude that the use of buffers at relays improves not only throughput but ironically the average end-to-end packet delay. Through extensive simulations, we validate our analytical results for the system when the data arrival and channel condition processes follow Bernoulli distribution. Furthermore, via the simulations under the settings of practical systems, we confirm our intuition for the general scenarios.