The patriarchal clan system is an important prerequisite for the formation and development of Chinese traditional culture. The spatial layout and space usage of traditional buildings are intimately related to patriarchal culture. Thus, analyzing the spatial layout and usage is an effective way to trace the culture of traditional buildings. In this study, a typical traditional building named Huzhai in eastern Zhejiang was examined as an example. The spatial layout characteristics of Huzhai and different space usage relations corresponding to different users under the influence of the patriarchal clan system were investigated through “all lines” analysis in space syntax. In this process, traditional ritual activities were considered crucial for tracing the culture of traditional buildings in the eastern Zhejiang province. The results demonstrate that spatial layout and the usage of traditional buildings in the eastern Zhejiang province have led to “class” distinctions under the influence of patriarchal culture. The sacrificial activities of families further emphasize the class distinctions of building space. The differences in building space usage among different classes reflect the unequal distribution of social resources in China’s traditional feudal society. These differences reflect the inequality of space mastership and control among different classes that are a result of the unequal distribution of social resources in China.