In this paper, we consider an initial [Formula: see text] random matrix with non-Gaussian correlated entries on each row and independent entries from one row to another. The correlation on the rows is given by the correlation of the increments of the Rosenblatt process, which is a non-Gaussian self-similar process with stationary increments, living in the second Wiener chaos. To this initial matrix, we associate a Wishart tensor of length [Formula: see text]. We study the limit behavior in distribution of this Wishart tensor in the high-dimensional regime, i.e. when [Formula: see text] are large enough. We prove that the vector corresponding to the [Formula: see text]-Wishart tensor converges to an [Formula: see text]-dimensional non-Gaussian vector, with Rosenblatt random variables on its hyper-diagonals and zeros outside the hyper-diagonals.