Nonlinear oscillations and its applications in physics, chemistry, engineering, biophysics, communications are studied with some analytical, numerical and experimental methods. In the present paper, hysteresis, resonant oscillations and bifurcation mode of a system modeled by a forced modified Van der Pol-Duffing oscillator are considered. The plasma oscillations are considered and are described by a nonlinear differential equation. By using the harmonic balance technique and the multiple time scales methods, the amplitudes of the forced harmonic, superharmonic and subharmonic oscillatory states are obtained. Then, we derived admissible values of the amplitude of the external strength. Some bifurcation structures and transition to chaos of the model have been investigated. The model presented several dynamics motions which are influenced by nonlinear parameters. It can be concluded that the nonlinear parameters have a real impact on the dynamics of the model.