With the widespread use of unmanned aerial vehicles (UAVs) in life, the real-time recognition of UAVs has become an important issue. The authors of this paper mainly studied the application of the random forest (RF) algorithm in the outdoor real-time recognition of UAVs. Mel-Frequency Cepstral Coefficient (MFCC) features were extracted from sound signals firstly, and then the RF method was combined with weighted voting to obtain the improved random forest (IRF) method to identify UAV sounds and environmental sounds. An experimental analysis was conducted. The modeling time of the IRF method increased by 9.52% compared with the RF method, and the recognition rate of the IRF method decreased with the increase of the distance from the microphone; however, the recognition rate of the IRF method was always higher than that of the RF method, and the recognition rate of the IRF method for the mixed samples was always higher than 90%. When the distance was 10 m, the IRF method still had a recognition rate of 91.29%. The experimental results verify the effectiveness of the IRF method for the outdoor real-time recognition of UAVs and its practical application feasibility.