A time-delayed response of individual living organisms to information exchange within flocks or swarms leads to the formation of complex collective behaviors. A recent experimental setup by Khadka et al., Nat. Commun. 9, 3864 (2018), employing synthetic microswimmers, allows to realize and study such behavior in a controlled way, in the lab. Motivated by these experiments, we study a system of N Brownian particles interacting via a retarded harmonic interaction. For N ≤ 3, we characterize its collective behavior analytically via linear stochastic delay-differential equations, and for N > 3 by Brownian dynamics simulations. The particles form nonequilibrium molecule-like structures which become unstable with increasing number of particles, delay time, and interaction strength. We evaluate the entropy fluxes in the system and develop an approximate time-dependent transition-state theory to characterize transitions between different isomers of the molecules. For completeness, we include a comprehensive discussion of the analytical solution procedure for systems of linear stochastic delay differential equations in finite dimension, and new results for covariance and time-correlation matrices.