| 01/08 |
L1 - Autodiff |
1-autodiff.pdf |
Link
|
Assignment 1 Release |
| 01/13 |
L2 - Deep Learning Compiler |
2-triton.pdf |
Link
|
|
| 01/15 |
L3 - FlashAttention |
3-flashattention.pdf |
Link
|
|
| 01/20 |
L4 - Data Parallelism |
4-pytorchddp.pdf |
Link
|
|
| 01/22 |
L5 - ZeRO |
5-zero.pdf |
Link
|
Assignment 1 Due; Assignment 2 Release |
| 01/27 |
NO CLASS (Severe Weather) |
|
|
|
| 01/29 |
L6 - Tensor Model Parallelism |
6-ptdp.pdf |
Link
|
Project Proposal Due |
| 02/03 |
L7 - Automated Parallelism |
7-alpa.pdf |
Link
|
|
| 02/05 |
L8 - Continuous Batching |
8-orca.pdf |
Link
|
Assignment 2 Due; Assignment 3 Release |
| 02/10 |
L9 - Paged Attention |
9-pagedattention.pdf |
Link
|
|
| 02/12 |
L10 - Disaggregating Prefill and Decoding |
10-distserve.pdf |
Link
|
Project Checkpoint 1 Report Due |
| 02/17 |
L11 - Parameter-Efficient Fine-Tuning |
11-qlora.pdf |
Link
|
|
| 02/19 |
L12 - Mixture of Experts |
|
Link
|
Assignment 3 Due; Assignment 4 Release |
| 02/24 |
L13 - Reinforcement Learning |
|
Link
|
|
| 02/26 |
L14 - Speculative Decoding |
|
Link
|
|
| 03/03 |
L15 - Sparse Attention |
|
Link
|
|
| 03/05 |
L16 - Fault Tolerance |
|
Link
|
Assignment 4 Due; Assignment 5 Release |
| 03/10 |
NO CLASS (Spring Break) |
|
|
|
| 03/12 |
NO CLASS (Spring Break) |
|
|
|
| 03/17 |
L17 - Tensor Processing Units |
|
Link
|
|
| 03/19 |
L18 - Multi Modalities |
|
Link
|
Project Checkpoint 2 Report Due |
| 03/24 |
L19 - Video Generation |
|
Link
|
|
| 03/26 |
L20 - Course Summary and Future Directions |
|
|
Assignment 5 Due |
| 03/31 |
Project Presentation |
|
|
|
| 04/02 |
Project Presentation |
|
|
|
| 04/07 |
Project Presentation |
|
|
|
| 04/09 |
Project Presentation |
|
|
|
| 04/14 |
NO CLASS (Finish Project Report) |
|
|
Project Final Report Due |