Refrigeration system evacuation is one of the most important steps in achieving refrigeration or air conditioning system quality. This procedure is used to remove water vapor and non-condensable gas from the pipeline and the heat exchangers that can be dangerous for a vapor compression refrigeration system. In this paper, the performance of an air conditioner (AC) was experimentally tested under varied duration of system evacuation from 0 to 60 minutes with an increment of 15 minutes. Cooling capacity, power consumption, and coefficient of performance were tested in accordance with ISO 5151 and ISO 16358-1. Generally, the ISO 16358-1 gives the lower power consumption and higher cooling capacity and coefficient of performance. ISO 16358-1 test results in 12.8% lower power consumption than that of ISO 5151. On the contrary, ISO 16358-1 test gives higher cooling capacity and coefficient of performance by 11.5% and 6.2%, respectively. It is also observed from this study that the longest evacuation period results in the most efficient operation, thus providing less effect on climate change and less environmental impact. However, it should be observed in a longer period of operation.