Dry Microwave Imaging (MWI) systems are more practical, hygienic and fast to operate since they do not require immersion liquid. However, the dielectric contrast between air and the part of the body under examination is larger, causing larger refraction effects. Including refraction in the image reconstruction algorithm significantly increases the computational effort, especially when imaging non-uniform shapes. Hence, our systematic study aims to evaluate the impact of neglecting refraction effects on MWI by using quantitative metrics and define objective guidelines that are lacking in the literature. We perform comparative studies with a spherical numerical phantom (which is typically used to represent simplified breast or head phantoms) by varying the phantom relative permittivity values between 4 and 40, metallic targets diameter between 5 and 15 mm, and the number of probing antennas. Additionally, the refraction effects are evaluated with anthropomorphic body phantoms representing a breast and the axillary region. We numerically and experimentally show that refraction tends to have greater impact on imaging results when phantom relative permittivity values exceed 8, while it has a minor effect in the remaining tested cases. This favours potential fast real-time image reconstruction. This study provides useful criteria to decide whether refraction should be considered or not for imaging reconstruction when developing new dry medical MWI setups.