Quadrotor fixed-wing hybrid vertical take-off and landing (VTOL) UAV, the so-called Quad-Plane, is susceptible to wind disturbance during the VTOL mode, which reduces its stability and limits its application. In order to dynamically analyze its response characteristics to wind disturbance and effectively optimize the wind disturbance rejection performance in VTOL mode, a dynamic flight simulation system based on the 6-DoF equations is established, in which the aerodynamic model of the Quad-Plane is established based on the wind tunnel test, the axial and lateral forces and moments of the rotor in wind disturbance are obtained through CFD simulation, and the Brushless DC (BLDC) motor, wind disturbance, and flight controller are modeled based on actual characteristics. The simulation system is verified and validated by comparing the simulation results with test results of a 5 kg real Quad-Plane. Using this system, the response characteristics of the Quad-Plane to the discrete and continuous wind disturbance are studied. In addition, the influences of major conceptual layout parameters that affect the wind disturbance rejection performance, such as take-off weight, weight distribution, Center of Gravity (CG) position, rotor incline angle, arm length, and arm angle, are studied. Finally, the approaches on designing a Quad-Plane with better wind disturbance rejection performance during the conceptual layout design are presented.