Tuned mass damper (TMD) is a single‐terminal damper, while inerter is a two‐terminal one, which are effective control devices. So, a hybrid damper with a series‐parallel inerter system and a TMD (SPIS‐TMD) in series is proposed. The main work of the manuscript is as follows. Firstly, based on the mechanical diagram of SPIS‐TMD and its equipment on the roof of a high‐rise structure, the general form of seismic motion equation was derived using dynamic finite element technology. Secondly, based on the method of quadratic decomposition for power spectrum density function (QD‐PSDF), the concise analytic solutions for zero‐, first‐, and second‐order response spectral moments (ZFSO‐RSMs) of the general response of structure with SPIS‐TMD were deduced. Thirdly, in response to the numerous parameters that affect the safety of high‐rise structures and the varying difficulty for obtaining SPIS‐TMD’s parameters, an optimization analysis technique was proposed, which is constrained by dynamic reliability and SPIS‐TMD’s parameter weights. Finally, three examples were given and results show the following. (1) The proposed analytical solutions for ZFSO‐RSMs are correct and high efficiency and can be extended to analysis of stationary seismic responses of general linear structures. (2) Calculating results of ZFSO‐RSMs of general responses with the number of vibration modes corresponding to a participation weight of 100% is exactly the same as those with all vibration modes, and the calculating time of the case of the participation weight of 100% is less than 1/12 that of the case of all vibration modes. (3) The failure probabilities of the structure with SPIS‐TMD with optimal parameters using the proposed method (with a limit failure probability of 0.1587), the structure only equipped with TMD with the same TMD’s parameters as SPIS‐TMD, and the structure without dampers are 0.1570, 0.7060, and 0.9778, respectively. It indicates that under the same conditions, the hybrid damper SPIS‐TMD has a better damping effect than a single TMD.