Reinforcement Learning Pong AI Agent

Pong AI Agent Demo.

Overview

I have implemented a reinforcement learning (RL) pong agent, using a Deep Q Network. The game uses Pygame for visualisation and PyTorch. Key Features include:

• •
  Speed Control: The agent can control its speed and position by moving up and down each lane as well as how quickly.
• •
  State Normalisation: Paddle and ball, speeds and positions are scaled to [0,1] for better DQN learning.
• •
  Rewards: Rewards are given for scoring.
• •
  Visualisation: Displays episode/reward and gameplay using Pygame.

This project is designed to run on either CPU or GPU.

DQN Model (model.py, agent.py)

• •
  QNetwork(model.py)
• •
  Architecture: 3-layer MLP (stateSize=5 → 128 → 64 → actionSize=3)
• •
  Activation: ReLU for hidden layers.
• •
  Device: Moved to GPU/CPU based on device.
• •
  ReplayBuffer(model.py)
• •
  Size: Stores up to 10,000 transitions (state, action, reward, nextState, done).
• •
  Samples: Batches of 64 for training.
• •
  DQNAgent(agent.py)
• •
  Epsilon greedy exploration: ε: 1.0 → 0.01, decay=0.995
• •
  Loss Function and Optimiser: Uses MSE Loss and Adam Optimiser(lr=0.001)

Improvements

The agent struggles to deal with the ball being hit at just before the corner so it has to change direction rapidly. Also, the model struggles with the initial start of the game, where the ball is moving directly horizontally.

Available here.

PyTorch DQN Tutorial here.