With the development of autonomous driving, future traffic will be composed of various participants. Integrating autonomous vehicles into the traffic flow composed of various types of traffic participants and minimizing conflicts between them is a critical research issue. Thus, this study presents a layered game-theoretic decision-making framework with situational awareness for autonomous vehicles, enabling adaptive decisions for autonomous vehicles in scenarios with multiple traffic participants of different driving characteristics. This paper’s situational awareness layer recognizes multiple participants’ politeness levels through their behavior and spatiotemporal relationships, allowing for a quantitative evaluation of their driving characteristics. The decision-making layer, built on Stackelberg game, adjusts the estimated cost of other traffic participants based on recognized politeness levels. The predictions of optimal behavior for traffic participants are obtained by minimizing the cost, according to which the optimal decision for the ego vehicle can be obtained. Besides, a set of parameters is used to construct the optimization problem as a convex optimization problem, so that the uniqueness of leader’s prediction of follower’s optimal action in each game can be guaranteed. To verify the feasibility and effectiveness, a trajectory planning layer for the autonomous vehicle is designed, the geometric safety constraint consists of planned trajectory and predicted trajectory of traffic participants are built to prevent collisions. Results indicate that the proposed framework can achieve balanced performance when interacting with traffic participants of different politeness levels.