Reinforcement Learning

A 3‑ECTS hybrid course covering Reinforcement Learning from basics to state-of-the-art algorithms. Students learn to formulate and solve real-world problems using classical and modern techniques, from Q-learning to PPO and SAC, with practical focus on algorithm selection and reward learning for contemporary applications like LLM training.

What You Will Learn

Students will understand the goals of Reinforcement Learning (RL) and the fundamental differences between RL and Supervised Learning. They will recognize the characteristics of problems where RL can be effectively applied and learn to formulate real-world problems as RL problems.

The course covers basic RL methods including Q-learning and its variants, providing a foundation for more advanced techniques. Students will learn off-policy Deep RL algorithms for discrete action spaces, specifically Deep Q-Networks (DQN). For continuous action spaces, the curriculum includes on-policy Deep RL algorithms such as REINFORCE, Actor-Critic, and Proximal Policy Optimization (PPO), as well as off-policy algorithms including Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), and Soft Actor-Critic (SAC).

By the end of the course, students will be able to select appropriate algorithms for specific problem types and apply RL techniques in scenarios where the reward function is unknown or must be learned.

Learning objectives:

• Understand the goals of Reinforcement Learning (RL) and the differences with Supervises Learning
• Understand characteristics of problems where RL can be applied and shine
• Learn to formulate a problem as a RL problem
• Learn basic methods for RL: Q-learning and variants
• Learn off-policy Deep RL algorithms for discrete action spaces: DQN
• On-policy Deep RL algorithms for continuous action spaces: Reinforce, Actor Critic and PPO
• Off-policy Deep RL algorithms for continuous action spaces: DDPG, TD3 and SAC
• Learn which algorithm to apply for a given problem
• Learn to apply RL when reward function is unknown

Agenda

Agenda: 6 weeks

• Week 1 – Introduction to RL
• Week 2 – Basic RL algorithms
• Week 3 – Deep RL
• Week 4 – Actor Critic approaches
• Week 5 – Practical RL
• Week 6 – Learning the reward function

Technical setup

Laptop with internet access; access to open‑source tools; access to Google Colab.