Streamflow and sediment runoff are important indicators for the changes in hydrological processes. In the context of environmental changes, decreases in both streamflow and sediment (especially in the flood season) are often observed in most of the tributaries of the middle Yellow River in China’s Loess Plateau. Understanding the effect of human activities could be useful for the management of soil and water conservation (SWC) and new constructions. In this paper, changes in streamflow and sediment during the flood season (June–September) of the 1966–2017 period in a typical loess hill and gully landform basin were analyzed. Basin-wide rainfall of the flood season decreased nonsignificantly with an average rate of −0.6 mm/flood season for the whole study period by using the trend-free pre-whitening based Mann–Kendall trend test, while the decreasing rate was weakened on the time scale. A remarkable warming trend (1985–1999) and two decreasing trends (1966–1984 and 2000–2017) were observed, and the overall increasing trend could be found in air temperature series with a rate of 0.01 °C/flood season during the study period. Statistical models were developed to describe the rainfall-runoff and rainfall-sediment processes in the pre-impact period (when the hydrological series was stationary). Furthermore, the relative effects of climate variability and human activities on hydrological changes were quantified. Results proved the dominant role of human activities (versus climate variability) on the reductions of both streamflow and sediment load. The relative contribution of human activities to streamflow decrease was 84.6% during the post-impact period 1995–2017, while the contributions were 48.8% and 80.1% for two post-impact periods (1982–1996 and 1997–2017), respectively, to the reduction of sediment load. Besides, the effect of the exclusion of anomalous streamflow or sediment events on change-point detection was also analyzed. It indicated that the anomalous events affect the detection of change points and should be given full consideration in order to decide whether to remove them in the change-point detection. Otherwise, the full series with anomalous samples will completely affect the attribution results of hydrological changes. We also suggest that large-scale SWC measures with different construction quality and operational life could intercept and relieve most floods and high sediment concentration processes, but may amplify the peaks of streamflow and sediment when the interception capacities are exceeded under the condition of extreme rainstorm events.