Generating a dataset for learning setplays from demonstration

Abstract

Coordination is an important requirement for most Multiagent Systems. A setplay is a particular instance of a coordinated plan for multi-robot systems in collective sports. Setplays are usually designed by robotics specialists using some existing tools, like the SPlanner, or by hand-coding. This work presents recent improvements to the Strategy Planner (SPlanner) and its corresponding FCPortugal Setplays Framework (FSF) to provide sophisticated setplays. This toolkit is useful to design strategic plans for robotic soccer teams as a particular case of Multi-Agent Systems (MASs). The new enhancements enable more realistic setplays, including, but not limited to, the definition of better pass strategies and defensive setplays. The enhanced tool is used to populate a dataset with demonstrations made by soccer experts and used in a Learning from Demonstration (LfD) approach to allow robotic soccer teams to learn new setplays. A new demonstration mode in the RoboCup Soccer Simulation 3D (SSIM3D) viewer RoboViz was also introduced to integrate this tool with SPlanner. Domain experts can use this set of tools to capture a specific scene in a game in RoboViz and use it as an initial step for a new setplay recommendation in SPlanner. The resulting dataset is organized into fuzzy clusters to be used in a reinforcement learning strategy. This paper describes the whole process. Article Highlights This paper’s main contribution is generating a dataset of setplays to support learning from demonstration in robotic soccer. A set of new features were added to the Strategic Planner(SPlanner) to enable the design of more realistic setplays. The official RoboCup viewer (Roboviz) was integrated with SPlanner using a new demonstration mode.

BahiaRT Setplays Collecting Toolkit and BahiaRT Gym

“…By default, SPlanner allows the creation of new setplays without providing any participant context or game state. The version presented in the BahiaRT Setplays Collecting toolkit was modified to support Demo Mode and new behaviors and features [3,4].…”

“…The BahiaRT Setplays Collecting Toolkit integrantes RoboViz and SPlanner to allow users to watch several game logs and generate setplays demonstrations that can be submitted to compose a setplays dataset to be used in reinforcement learning approaches [5,4].…”

“…The file must be exported to the "setplays" folder, as shown in the Figure 6. Finally, to send the setplays to the setplays dataset organized during this research [5,4], users must save them in a .zip file (Figure 7) and then send them through the "Submission Form".…”

“…OpenAI Gym allows agents to use several environments for reinforcement learning training. Though the toolkit already provides many pre-built environments, none of those was ideal for RoboCup community, nor could be used with the RCSSServer3d(3D soccer simulation server) 4 . Therefore, we decided to create a customized environment from scratch to train agents on the RoboCup Soccer Simulation Server.…”

“…The BahiaRT Setplays Collecting Toolkit supports the crowdsourcing strategy used to build a setplays demonstrations dataset [5,4]. Most research in machine learning deals with the difficulty of building realistic datasets using the experts' experience.…”