Ground Source Heat Pumps (GSHPs) take advantage of the high thermal inertia of the ground to achieve a higher energy efficiency compared to Air Source Heat Pumps. GSHPs, therefore, have the potential to reduce heating, cooling, and domestic hot water costs, however the high installation cost of borehole heat exchangers (BHEs) limits the growth of such installations. Nevertheless, GSHPs can be profitable under certain conditions (climate, expensive fuels, subsidies, etc.), which can be identified using geo-referenced data and Geographical Information Systems (GIS). The proposed work investigates the economic and financial ability of GSHPs to cover the heat demand of the residential building stock of the Italian region Valle d’Aosta. To identify the opportunities offered by GSHPs in the Valle d’Aosta region, more than 40,000 residential buildings were analyzed using a GIS-based method. The return on the investment was then assessed based on the occurrence of two conditions—the Italian subsidies of the “Conto Termico” and the installation of rooftop photovoltaic (PV) systems—which contribute to the reduction of the initial and operation costs, respectively. The life-cycle costs of the four resulting combinations were compared with conventional systems composed of an oil/gas boiler and an air-source chiller. One of the main findings of this study is that subsidies exert a key role in the financial feasibility of GSHPs, especially for replacing gas boilers, whereas the presence of a PV system has a minor influence on the financial analysis carried out.