Inside a closed many-body system undergoing the unitary evolution, a small partition of the whole system exhibits a local relaxation. If the total degrees of freedom of the whole system is a large but finite number, such a local relaxation would come across a recurrence after a certain time, namely, the dynamics of the local system suddenly appear random after a well-ordered oscillatory decay process. It is found in this paper, for a collection of N two-level systems (TLSs), the local relaxation of one TLS within has a hierarchy structure hiding in the randomness after such a recurrence: similar recurrences appear in a periodical way, and the later recurrence brings in stronger randomness than the previous one. Both analytical and numerical results that we obtained well explains such hierarchy recurrences: the population of the local TLS (as an open system) diffuses out and regathers back periodically due the finite-size effect of the bath [the remaining (N − 1) TLSs]. We also find that the total correlation entropy, which sums up the entropy of all the N TLSs, approximately exhibit a monotonic increase; in contrast, the entropy of each single TLS increases and decreases from time to time, and the entropy of the whole N-body system keeps constant during the unitary evolution.