The phase inversion method, a technical measurement procedure, is often used to evaluate the performance of noise reduction algorithms in hearing aids. However, a detailed comparison of these technical measurements with the perceived loudness is missing. Therefore, commercially available hearing aids of six different manufacturers were evaluated technically and in a study with 18 normal-hearing listeners. First, the output signals of the hearing aids with and without activated noise reduction were recorded in a test box. Then, the test subjects evaluated the perceived loudness of these recordings within multiple two alternative forced choice (2-AFC) tasks. During one task, the test subjects had to focus either on the speech or noise signal and were asked to select the louder of two signals, which both contained a mixture of speech and noise. These results provide not only the perceived SNR but also the perceived speech and noise levels. Comparing the results of the 2-AFC tasks and the phase inversion method basically shows good agreement. Nevertheless, a simple computation of the sound pressure level can lead to significant deviations. Therefore, another possibility for the analysis of the results of the phase inversion method to better match the perceived loudness is presented.