Purdue CS 55800ROB Introduction to Robot Learning (Spring 2024)

This course covers topics in motion planning, estimation, and control to design algorithms for robots to safely interact with their environments and perform various challenging tasks under constraints. The first part of this course focuses on classical techniques such as search-based and sampling-based planning, PID control, Model Predictive Controller (MPC), and Bayesian filters. The second part covers modern deep learning and deep reinforcement learning techniques and their application to planning and decision-making in robotics.

Administrative Information

Instructor: Ahmed H. Qureshi (ahqureshi@purdue.edu) | Office Hours & Location: By appointment
TA: Zixing Wang (wang5389@purdue.edu) | Office Hours: Monday 2:30-3:30 at HAAS 143 table 1 or by appointment
Time & Location: MWF 12:30-1:20, LWSN 1106

Course Content

Sr	Date	Topics (Tentative)	Readings
1	1/8	Introduction to Robotics
2	1/10	Robot Manipulators (Forward/Inverse Kinematics)
3	1/12	Collision Avoidance & Motion Planning I: Search-based Methods (A*)
4	1/15	MLK Holiday
5	1/17	Motion Planning II: Sampling-based methods I (RRT, PRM)	[1] [2]
6	1/19	Motion Planning III: Sampling-based methods II (RRTConnect, RRT*)	[1] [2]
	1/19	Assignment 1 (Release)
7	1/22	Motion Planning IV: Sampling-based methods III (B-RRT, IB-RRT, Informed-RRT*)
8	1/24	Motion Planning V: Advance Constraints (Kinodynamic, Manifolds, etc.)	[1]
9	1/26	Motion Planning VI: Kinodynamic Constraints
10	1/29	Optimal Control I: PID I
11	1/31	Optimal Control II: PID II
12	2/2	Project Discussions
	2/2	Assignment 1 (Due 11:59pm (EDT))
	2/2	Assignment 2 (Release)
13	2/5	Optimal Control III: MPC I	[1]
14	2/7	Optimal Control IV: MPC II
15	2/9	Optimal Control V: MPC III
16	2/12	Estimation I: Probability Review, Bayes Filtering
17	2/14	Estimation II: Kalman Filter	[1]
18	2/16	Estimation III: Extended Kalman Filter
	2/16	Assignment 2 (Due 11:59pm (EDT))
19	2/19	ML for Motion Planning I: Motion Planning Networks	[1][2]
	2/19	Project title and team members (Due 11:59pm (EDT))
20	2/21	ML for Motion Planning II: Learning Sampling Distributions I
21	2/23	ML for Motion Planning III: Learning Sampling Distributions II	[1]
	2/23	Assignment 3 (Release)
22	2/26	ML for Motion Planning IV: Constrained Motion Planning	[1]
23	2/28	ML for Motion Planning V: Invited Talk
24	3/1	Deep Reinforcement Learning I: MDPs, Value Function, Q-Function
25	3/4	Deep Reinforcement Learning II: Dynamic Programming, MCMC, TD-Learning
26	3/6	Deep Reinforcement Learning III: Policy Gradients I (Reinforce)
27	3/8	Deep Reinforcement Learning IV: Vanilla Policy Gradient
	3/8	Assignment 3 (Due 11:59pm (EDT))
28	3/11	Spring Break
29	3/13	Spring Break
30	3/15	Spring Break
	3/18	Project Milestone 1 (Due 11:59pm (EDT))
	3/18	Assignment 4 (Release)
31	3/18	Deep Reinforcement Learning V: Advance Policy Gradients Methods I (TRPO, PPO)
32	3/20	Deep Reinforcement Learning VI: Advance Policy Gradients Methods II (DDPG, A3C)
33	3/22	Invited Talk: Ruiqi Ni - Physics-informed Neural Networks for Motion Planning
34	3/25	Deep Reinforcement Learning VII: Advance Policy Gradients Methods III (TD3, SAC)
35	3/27	Imitation Learning I
36	3/29	Imitation Learning II
	3/29	Assignment 4 (Due 11:59pm (EDT))
37	4/1	Inverse Reinforcement Learning I
38	4/3	Inverse Reinforcement Learning II
39	4/5	Final Exam
40	4/8	-
41	4/10	Invited Talk: Zixing Wang - DeRi-Bot: Learning to Collaboratively Manipulate Rigid Objects via Deformable Objects
	4/11	Project Milestone 2 (Due 11:59pm (EDT))
42	4/12	Project Presentations I
43	4/15	Project Presentations II
44	4/17	Project Presentations III
45	4/19	Project Presentations IV
46	4/22	Project Presentations V
47	4/24	Project Presentations VI
48	4/26	Project Presentations VII
	4/29	Project Milestone 3: Project Report, Demonstration Videos, Codes, Trained Models & Datasets(Due 11:59pm (EDT))

Prerequisites

Graduate standing or a “C” or better in: CS 47100 AND (STAT 35000 or STAT 35500) AND (MA 26500 or MA 35100)

Textbooks

Siciliano, B., Khatib, O., & Kröger, T. (Eds.). (2008). Springer handbook of robotics (Vol. 200). Berlin: springer. Additional material will be selected from other sources which may include:

Principles of Robot Motion (Theory, Algorithms, and Implementations) by Howie Choset, Kevin M. Lynch, Seth Hutchinson, George A. Kantor, Wolfram Burgard, Lydia E. Kavraki and Sebastian Thrun.
Reinforcement learning: An introduction by Richard S. Sutton, and Andrew G. Barto. MIT press, 2018.
Deep Learning by Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. MIT press, 2016.

Grading Policy

Assignments (4 X 10%)	40%
Final Exam	23%
Project	35%
Class Participation	2%

Project total	35%
Milestone 1: Planning/Control component + Project proposal	10%
Milestone 2: Learning component + Initial results + Project presentation	10%
Milestone 3: Full Project Report + Demo videos + Code with documentation	15%

Project: Students can do solo project or as a team of two (max).

Project proposal and final report template:https://www.overleaf.com/latex/templates/ieee-journal-paper-template/jbbbdkztwxrd

Late Submission Policy: All deadlines are firm unless notified in advance. Late submissions can only be accepted within next 48hr (1second==48hr) of the deadline but will result in a straight 25% off.

Academic Integrity

This course defaults to Purdue standards on intellectual integrity and academic conduct. Therefore, students are responsible for reading the following pages and comply with them throughout this course.