CompSci 590.7
Reinforcement Learning

• Overview

• Schedule

• Assignments

• Grading

Meeting Schedule

Important notes:

• Links on the on the left are broken but will be fixed shortly!
• This is an ambitious schedule that will probably slip, but there is time built in to the end to accomodate this.
• Unless otherwise stated, chapters refer to Algorithms for Decision Making
• SB refers to the Sutton and Barto text

Date	Topic	Homework	Slides	Supplemental Material
01/07/26	Introduction	Read chapter 1	intro	SB chapter 1
01/12/26	Probability and Simple Decisions	Read chapters 2 and 6	probability SimpleDecisions	Foundations of Computer Science Sections 4.9-4.12
01/14/26	Simple Decisions
01/21/26	Algorithms for MDPs	Read chapter 7	MDPs	SB chapters 3 and 4
01/26/26	Finish Algorithms for MDPs	HW1a assigned, due 2/6
01/28/26	Approximations and Search	Read chapters 8 and 9	ApproxVI	Stable Function Approximation in Dynamic Programming
02/02/26	Approximations and Search (continued)		TreeSearch
02/04/26	Search		TreeSearch
02/09/26	Model Free RL	Read chapter 17	ModelFreeRL	SB chapter 6
02/11/26	Advanced Model Free RL	Read Human Level Control Through Deep Reinforcement Learning HW2 due 02/27/26	DeepRL	SB Chapter 9 David Silver's Slides
	Bandits	Read chapter 15	bandits	SB Chapter 2 Introduction to Multi-Armed Bandits by Aleksandrs Slivkins
	Model Based Reinforcement Learning	Read chapter 16	ModelBasedRL
	Sarsa, Lambda	Read chapters 10 and 11		SB Chapters 7,12
	Policy Search
	Policy Gradient	Read chapters 12 and 13
	Finish Policy Gradient, Review Linear Programs
	Learning From Demonstration	Read Chapter 18
	Reproducibility, Shaping and Catch up	Read chapture 17.5, Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping,		Potential Shaping and Q-value Initialization are Equivalent Measuring the Reliability of Reinforcement Learning Algorithms
	Hidden Markov Models and Particle Filters	Read chapter 19
	POMDP basics	Read chapters 19 and 20		POMDPs for Dummies
	POMDPs (approximate solutions)	Read chapters 21, 22 and 23
	Matrix Games	Read chapter 24
	Markov Games	Read chapter 26, 27
	Catch up, extra topics (abstraction, hierarchy, etc.), projects