One of the main factors limiting the productivity of potatoes (Solanum tuberosum L.) is water stress. Two irrigation systems: full irrigation (I) and rainfed conditions (R), were compared over the growing seasons from 2012–13 to 2019–20. The evaluated varieties were Desiree, Karú-INIA, Patagonia-INIA, Puyehue-INIA, Yagana-INIA, Yaike, and Porvenir. This study determined (i) the yield and tuber size distribution, (ii) their relationship between productivity and environmental conditions, and (iii) the most drought-tolerant varieties based on drought tolerance indices. Nine indices including yield index (YI), tolerance index (TOL), mean productivity (MP), geometric mean productivity (GMP), harmonic mean (Harm), stress tolerance index (STI), harmonic mean productivity (HMP), yield reduction (Yr), and stress susceptible index (SSI) were calculated by using tuber yield under I and R conditions. Tuber yield under R conditions decreased by 27 and 34%. However, the highest yield under R conditions occurred in years with more precipitation between 60 and 120 days after planting (DAP; ±60 mm). Under R conditions, the varieties Porvenir, Patagonia-INIA, Yaike, and Puyehue-INIA showed more tolerance to water stress. Water stress negatively affected tuber size distribution, reducing the production of tubers with size >65 mm by 50–60%. The best indices to study drought tolerance were TOL, MP, GMP, Harm, STI, and HMP. This study suggests that in southern Chile, an area with big yield potential, typically cultivated as rainfed, with cool temperate climate conditions and favorable soil properties for potatoes, as Andisols, available rainfall is still a constraint for yield. Therefore, using more water stress-tolerant varieties and providing supplementary irrigation between 60 and 120 DAP are critical to optimize yield and avoid the failure of the crop in years with remarkably low precipitations, which will be more pronounced in the future according to weather trends. These results exemplify how much we can lose in productivity in rainfed conditions even in one of the most favorable areas for growing potatoes in the world and how risky this situation can be for the performance of the potato farms in the future.