The unsafe behavior of construction workers is the most direct reason of frequent construction site accidents. In order to improve the safety management of construction sites and figure out the causal relationship among the influencing factors in the field of construction workers’ unsafe behaviors, literature research, questionnaire survey, decision-making trial, and evaluation laboratory—interpretive structural modeling—crossimpact matrix multiplication applied to classification (DEMATEL-ISM-MICMAC) methods were used in combination in this thesis. The analysis of collected data was carried out in three dimensions: individual, organizational environment, and safety management. A framework about influencing factors of construction workers’ unsafe behaviors was constructed. DEMATEL-ISM was used to construct the explanatory structural model, which to analyze the influence relationships and hierarchical relationships among factors. MICMAC method was used to analyze the driving dependency. ISM model consists of six parts, the bottom of which includes three influencing factors: work environment, safety supervision, and concernment of superior. With characteristics of high drive and low dependence, the bottom layer are the root causes of construction workers’ unsafe behaviors. Work environment and concernment of superior are the core indicators of it. The intermediate layer with a low drive and low dependence covers six factors: psychological status, physical health, professional skills, organizational climate, work quota, and safety plan. It is the indirect factor to influence construction workers’ unsafe behaviors. The top layer is composed of safety awareness, safety education, and technical delivery. Safety awareness is the core of individual dimension, showing the characteristics of low drive and high dependence, is the direct factor to influence construction workers’ unsafe behaviors. Based on the DEMATEL-ISM-MICMAC method, the methodology to reduce construction workers’ unsafe behaviors was proposed.