Strategising RoboCup in Real Time with Uppaal Stratego

The RoboCup simulator is a playing ground for Agents and Artificial Intelligence research. One of the main challenges provided by RoboCup is generating winning strategies for a set of agents playing soccer, given a partial and noisy view of the game state. Additionally, RoboCup is timing sensitive, and all decisions have to be sent to the server within each tick of 100ms. This paper presents a method for generating strategies by modelling players and scenarios as timed automata in the Uppaal environment. The newest extension of Uppaal, called Uppaal Stratego, allows for synthesising strategies optimising a reward function that is used to guide the decision process. In order to stay within the time frame of 100ms, two approaches were tested, namely forecasting the game state and generating a strategy asynchronously for a later point in time, and generating strategies beforehand and saving them in a lookup table. Four timed automata were developed, and were tested against publicly available methods. We found that strategies could be successfully generated and used within the time constraints of RoboCup using our proposed method. Especially when combined together, our strategies are able to outperform most published methods, but lose against the published world champions.