To accurately predict the optimum supplemental modal damping ratio of the cable and the corresponding size of the inertial mass damper (IMD), combined effects of the cable sag, the cable flexural rigidity, and the boundary conditions on the control performance of the cable with the IMD are well investigated in this refined study. An analytical model of the cable-IMD system considering these effects is developed. The equation of motion of the cable-IMD system is transformed into a complex eigenvalue problem through the finite difference method. Experimental results from a scaled cable model with an IMD are then used to verify theoretical solutions. Three typical cables in actual cable-stayed bridges are selected for case studies. The results show that the theoretically predicted modal damping ratios of the cable with an IMD, taking into account the sag and the flexural rigidity, agree well with those identified from experimental results, while would be often overestimated with a taut-cable model. Moreover, experimental damping ratios of the cable always fall between those theoretically calculated with fixed ends or pinned ends for each case. Finally, to be conservative in actual design, it is recommended to use the cable-IMD system model with fixed ends to calculate the required damper size and predict the resulting modal damping ratio of the cable, since the corresponding theoretical solution often gives the lower bound of supplemental damping ratio of the cable.