To identify hygrothermal transfer patterns of exterior walls is a crucial issue in the design, assessment, and construction of buildings. Temperature and relative humidity, as sensor monitoring data, were collected from the outside of the wall to interior bamboo and wood composite sheathing over the year in Huangshan Mountain District, Anhui Province, China. Combining the machine learning method of reservoir computing (RC) with agglomerative hierarchical clustering (AHC), a novel clustering framework was built for better extraction of the characteristics of hygrothermal transfer on the time series data. The experimental results confirmed the hypothesis that the change in the temperature and relative humidity of the outside of the wall (RHT12) dominated the change of the interior sheathing (RHT11). The delay time between two adjacent peaks in temperature was 1 to 2 h, while that in relative humidity was 1 to 4 h from the outside of the wall to interior bamboo and wood composite sheathing. There was no significant difference in temperature peak delay time between April and July. Temperature peak delay time was 50 to 120 min. However, relative humidity peak delay time was 100 to 240 min in April, whereas it was 20 to 120 min in July. The impact formed a relatively linear relationship between outdoor temperature and relative humidity peak delay time. The hygrothermal transfer patterns were characterized effectively by the peak delays. The discovery of the hygrothermal transfer patterns for the bamboo and wood composite walls using the machine learning method will facilitate the development of energy-efficient and durable bamboo and wood composite wall materials and structures.