Lateral-directional piloted simulation experiments were conducted in order to analyze pilot responses when the pilot controls an aircraft with only the ailerons after rudder failure. The experiments were carried out using a fixed-base flight simulator and four pilots. Six aircraft configurations, made by changing the stability derivatives, were tested, and pilot models were identified using the least-squares method. An analysis of the closed-loop shows that the closed-loop gain and stabilities correlate with the tracking error of the compensation task. Additionally, from the results of analysis, an evaluation function using pilot-in-the-loop with a fixed pilot model was developed. This function was able to predict the compensational error analytically from the airplane dynamics.