Football is one of the most popular sports in the world, and its competition has received increasing people’s attention. With the increasing number of robot football competitions, more strategic planning is needed for robot football matches. In a tournament, each player has their own task and must have the skills to complete it. In this paper, we use the RoboCup2D platform to give details of the server and client roles, introduce the agent model in RoboCup2D, and compare the plan design and scheme design presented in the current study using the SARSA algorithm, one of the augmented methods classified as TD learning metrics. In addition, heuristic information was introduced and implemented to enhance learning through the sharing of Q values between participants and reinforcement learning. A comparative analysis of the feasibility of the SARSA algorithm in the context of its application in RoboCup2D was carried out, and the experimental results proved that our algorithm was effective in improving the team’s offensive and defensive capabilities.