Training Game-Playing Agents using Reinforcement Learning

Over the past two decades, applications of Artificial Intelligence (AI) in virtual environments have rapidly increased. One kind of rich environment are commercial video games. Commercial video games as a learning environment are greatly beneficial since they have existing agents (e.g. characters, non-playable characters (NPC)) which can be turned into learning agents, a rich environment in which micro or macro goals (e.g. getting a high-score, and finishing a quest, respectively) can be set for an agent to optimize how to complete them and obtain a maximum reward. The major factor preventing the use of commercial video games as learning environments is that they are not open source. In the gaming business, the rate of innovation in AI is exponential. Building dynamic AI behaviours that require the agent to make sophisticated judgments is tough. Furthermore, hardcoding all adversary states is not technically feasible. Creating hundreds of alternative AI states for high-profile games, even if they are developed using finite state machines, would take a long time.

In this research we examine Unity Game Engine and Unity’s open-source ML-Agents Toolkit, since it is the testing environment chosen for experimentation. Unity is relatively easy to use and allows for creating a learning environment. Since Unity’s ML-Agents Toolkit is fairly new and is attracting interest, it is important to test the practical implementation of the toolkit and different aspect of it. For this reason, our project focuses on the usability of Unity’s ML-Agents Toolkit to make an agent learn to optimize its behaviour to accomplish a given task. We make use of more modern Reinforcement Learning algorithms provided by Unity ML-Agents Toolkit to create game-playing agents and test their behaviour in several agent learning environments created using Unity. We then compare and contrast the performance of these algorithms by analyzing the cumulative rewards w.r.t the steps the agent takes from beginning to end of training.

Software Requirements

1. Unity Editor 2021.2.0f1

2. Unity ML-Agents Toolkit Package 2.1.0

3. Microsoft Visual Studio / Visual Studio Code 

4. Pytorch with TensorBoard

Version Control

Plastic SCM