A unified design model is proposed for various kinds of passive dynamic absorbers (PDAs) attached to buildings with different lateral resisting systems. A total of five different PDAs are considered in this study: (1) tuned mass damper (TMD), (2) circular tuned sloshing damper (C-TSD), (3) rectangular tuned sloshing damper (R-TSD), (4) two-way liquid damper (TWLD), and (5) pendulum tuned mass damper (PTMD). The unified model consists of a coupled shear-flexural (CSF) discrete model with equivalent tuned mass dampers (TMDs), which allows the consideration of intermediate modes of lateral deformation. By modifying the nondimensional lateral stiffness ratio, the CSF model can consider lateral deformations varying from those of a flexural cantilever beam to those of a shear cantilever beam. The unified model was applied to a 144-meter-tall building located in the Valley of Mexico, which was subjected to both seismic and along-wind loads. The building has similar fundamental periods of vibration and different nondimensional lateral stiffness ratios for both translational directions, which shows the importance of considering both bending and shear stiffness in the design of PDAs. The results show a great effectiveness of PDAs in controlling along-wind RMS accelerations of the building; on the contrary, PDAs were ineffective in controlling peak lateral displacements. For a single PDA attached at the rooftop level, the maximum possible value of the PDA mass efficiency index increases as the nondimensional lateral stiffness ratio decreases; therefore, there is an increase in the vibration control effectiveness of PDAs for lateral flexural-type deformations.