A quantitative characterization of brain-wide activity imposes strong constraints on mechanistic models that link neural circuit connectivity, brain dynamics, and behavior. Here, we analyze whole-brain calcium activity in larval zebrafish captured by fast light-field volumetric imaging during hunting and spontaneous behavior. We found that the brain-wide activity is distributed across many principal component dimensions described by the covariance spectrum. Intriguingly, this spectrum shows an invariance to spatial subsampling. That is, the distribution of eigenvalues of a smaller and randomly sampled cell assembly is statistically similar to that of the entire brain. We propose that this property can be understood in the spirit of multidimensional scaling (MDS): pairwise correlation between neurons can be mapped onto a distance function between two points in a low-dimensional functional space. We numerically and analytically calculated the eigenspectrum in our model and identified three key factors that lead to the experimentally observed scale-invariance: (i) the slow decay of the distance-correlation function, (ii) the higher dimension of the functional space, and (iii) the heterogeneity of neural activity. Our model can quantitatively recapitulate the scale-invariant spectrum in zebrafish data, as well as two-photon and multi-area electrode recordings in mice. Our results provide new insights and interpretations of brain-wide neural activity and offer clues on circuit mechanisms for coordinating global neural activity patterns.