In this paper, a nonlinear vibration system with friction and linear and nonlinear springs is modeled and analyzed. The analysis examined how the combination of nonlinear variables affects the displacement of the system using the slowly varying amplitude and phase (SVAP) method. The break-loose frequency at which relative motion begins was obtained as a function of the friction ratio, and it was found that the displacement transmissibility differed depending on the change in design parameters. The displacement transmissibility response showed a unique phenomenon in which bifurcation occurred in the front resonant branch before the maximum response point when the linear damping coefficient was small and the friction coefficient was large, and the displacement transfer curve was separated at a specific parameter value. This phenomenon can be divided into three parameter zones considering the bifurcation pattern and stability of the displacement transmissibility curve. In addition, a 3-D spatial zone of dimensionless parameters was presented, which can predict stability during the design process, along with the drawing method and procedure. This can be conveniently utilized in the process of setting the parameters of the isolators considering the stability of the response during the design. In the analysis and design process of vibration isolators with friction damping, this study has important implications for practical applications.