[1] The magnitude of the Sun's influence on climate has been a subject of intense debate. Estimates of this magnitude are generally based on assumptions regarding the forcing due to solar irradiance variations and climate modeling. This approach suffers from uncertainties that are difficult to estimate. Such uncertainties are introduced because the employed models may not include important but complex processes or mechanisms or may treat these in too simplified a manner. Here we take a more empirical approach. We employ time series of the most relevant solar quantities, the total and UV irradiance between 1856 and 1999 and the cosmic rays flux between 1868 and 1999. The time series are constructed using direct measurements wherever possible and reconstructions based on models and proxies at earlier times. These time series are compared with the climate record for the period 1856 to 1970. The solar records are scaled such that statistically the solar contribution to climate is as large as possible in this period. Under this assumption we repeat the comparison but now including the period 1970-1999. This comparison shows without requiring any recourse to modeling that since roughly 1970 the solar influence on climate (through the channels considered here) cannot have been dominant. In particular, the Sun cannot have contributed more than 30% to the steep temperature increase that has taken place since then, irrespective of which of the three considered channels is the dominant one determining Sun-climate interactions: tropospheric heating caused by changes in total solar irradiance, stratospheric chemistry influenced by changes in the solar UV spectrum, or cloud coverage affected by the cosmic ray flux.