The behavior of construction workers has a significant impact on the overall safety climate of a project. The purpose of this paper is to figure out the evolutionary pattern of workers' unsafe behavior and to minimize its occurrence. We constructed a two-sided evolutionary game model consisting of workers and managers to explore the focal point of interest, strategy equilibrium conditions, and behavior evolution process. The experimental results of stability analysis and system dynamics show that there are two stable states in all four cases, (Safe behavior, Negative management) as well as (Unsafe behavior, Negative management). The lower the initial willingness of workers to behave unsafely, the faster they reach a safe steady state. By contrast, managers' strategy choices have a certain lag. Workers are discouraged from choosing unsafe behavior under both the positive incentive of raising bonuses and the negative incentive of raising fines. And the sensitivity of the two incentives is similar. For indirect effect risk loss, when it is effectively controlled during safe construction, workers quickly gravitate toward safe behavior. These findings provide a reference for construction safety management. Several practical suggestions were proposed from three perspectives: the worker, the manager, and the site safety climate, focus on the theme of reducing unsafe behavior and achieving a virtuous cycle of safety climate.