In this project, I used a DQN model to train an agent to play the Unity food collector environment.

This environment has 37 states with 4 actions:

• 0 - move forward.
• 1 - move backward.
• 2 - turn left.
• 3 - turn right.

This environment is episodic, and to solve it, the agent must get an average score of +13 over 100 consecutive episodes.

• Getting Started
• Runing the scripts
• Author
• License

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

I have tested this repo with Python 3.9 and 3.10. To continue, install either of these versions on your local machine. With Python installed, I suggest you create a virtual environment to install required libraries:

python -m venv desired_path_for_env

Activate this environment before moving to next step. For addirional help, check Python documentation here.

The required packages for this project are listed in requirements file. To install these libraries, from the repo folder, run the following command in your virtual env:

python -m pip install -r requirements.txt

The already built Unity environment for this project is accessible from following links:

• Linux: click here
• MacOS: click here
• Windows (32-bit): click here
• Windows (64-bit): click here

Decompress (unzip) the downloaded file and copy it to the repo folder.

The training and testing scripts are located in scripts folder.

To train the model, use train_agent.py script. This script accepts the following arguments:

• Path to downloaded Unity App: --unity-app
• Target Score to save trained model: --target-score

cd scripts
python train_agent.py --unity-app Banana.app --target-score 13

On my machine, the environment was solved in 481 episodes:

Episode 100     Average Score: 1.065
Episode 200     Average Score: 4.18
Episode 300     Average Score: 7.93
Episode 400     Average Score: 11.20
Episode 481     Average Score: 13.07
Environment solved in 481 episodes!     Average Score: 13.07
Trained model weights saved to: checkpoint_481.pth

Saved Trained Checkpoint

To compare a trained agent with a untrained one, use [test_agent.py] script. This script accepts the following arguments:

• Path to downloaded Unity App: --unity-app
• Path to saved model checkpoint: --checkpoint-file

cd scripts
python test_agent.py --unity-app Banana.app --checkpoint-file ../checkpoints/checkpoint_481.pth

• Sina Fathi-Kazerooni - Website

This project is open source under MIT License and free to use. It is for educational purposes only and provided as is.

I have used parts of scripts in Udacity DRL repo under MIT License. Scripts in dqn and mlagents are based on Udacity DRL repo with minor modifications.