The paper investigates the formation control problem for high-order linear swarm systems with limited communications such as time-varying delays and switching interconnections. Firstly, the problem description is given including the dynamics of high-order swarm systems, formation protocol and the definitions of formation maintenance and tracking. Secondly, four formation conditions are derived: the former three are related to formation function, reference trajectory, auxiliary functions and network topology; the fourth one is equivalent to the stability of N-1 time-delay systems. In order to get lower conservative criteria, the Free-weighting Matrices(FWM) approach is employed to analyze the stabilization problem. Finally, the allowance upper bound of delays is calculated through solving the feasible linear matrix inequalities (LMIs). Numerical examples and simulation results are given to demonstrate the effectiveness and benefit on reducing conservativeness of the proposed method. INDEX TERMS Formation tracking, formation maintenance, high-order swarm systems, time-varying delays, switching interconnections.