| Event Type | Date | Description | Course Materials |
|---|---|---|---|
| Lecture | Thursday, Sept. 2 | Intro to Deep Learning, historical context. |
[slides] [python/numpy tutorial] [jupyter tutorial] |
| Lecture | Tuesday, Sept. 7 | Image classification and the data-driven approach k-nearest neighbor Linear classification |
[slides] [image classification notes] [linear classification notes] |
| Lecture | Thursday, Sept. 9 | Loss Functions Optimization |
[slides] |
| Optional Discussion | Friday, Sept. 10 | Python setup, Google Colab, and the basics of Python | |
| Lecture | Tuesday, Sept. 14 | Backpropagation & Neural Networks I |
[slides] [backprop notes] [Efficient BackProp] (optional) related: [1], [2], [3] (optional) |
| Lecture | Thursday, Sept. 16 |
Neural Networks II Higher-level representations, image features Vector, Matrix, and Tensor Derivatives |
[slides] handout 1: Vector, Matrix, and Tensor Derivatives handout 2: Derivatives, Backpropagation, and Vectorization Deep Learning [Nature] (optional) |
| Optional Discussion | Friday, Sept. 17 | Reviewing the chain rule, applying the chain rule to vectors | [slides] |
| Lecture | Tuesday, Sept. 21 |
Neural Networks III Training Neural Networks I Activation Functions |
[slides] [Neural Nets notes 1] tips/tricks: [1], [2] (optional) |
| Lecture | Thursday, Sept. 23 |
Training Neural Networks II weight initialization, batch normalization |
[slides] [Neural Nets notes 2] [Batch Norm] Copula Normalization (optional) |
| Optional Discussion | Friday, Sept. 24 | Vector, Matrix, and Tensor Derivatives | |
| Lecture | Tuesday, Sept. 28 |
Training Neural Network III: babysitting the learning process, hyperparameter optimization |
[slides] [Bengio 2012] (optional) |
| Lecture | Thursday, Sept. 30 |
Training Neural Network IV: model ensembles, dropout |
[slides] [Neural Nets notes 3] LeNet (optional) |
| Lecture | Tuesday, Oct. 5 | DropOut... continued |
[slides] |
| Lecture | Thursday, Oct. 7 | Convolutional Neural Networks | [slides] |
| Optional Discussion | Friday, Oct. 8 | Batch normalization | |
| Lecture | Tuesday, Oct. 12 |
ConvNets for spatial localization Object detection |
[slides] ResNet (optional) FCN (optional) [Stanford cs231n project reports: spring 2017] [Stanford cs231n project reports: winter 2016] [Stanford cs231n project reports: winter 2015 |
| Lecture | Thursday, Oct. 14 |
ConvNets for spatial localization Object detection... continued |
|
| Lecture | Tuesday, Oct. 19 | Object detection... continued | A tool to visualize convolutions |
| Lecture | Thursday, Oct. 21 |
Understanding and visualizing Convolutional Neural Networks... continued |
[slides] [visualization notes] |
| Lecture | Tuesday, Oct. 26 | Backprop into image: Visualizations, deep dream | |
| Lecture | Thursday, Oct. 28 |
Neural Texture Synthesis and Style Transfer Creating Adversarial Examples |
[slides] [slides] |
| Lecture | Tuesday, Nov. 2 | Generative Adversarial Networks |
[slides] [Style-GAN] [Alias-Free GAN] |
| Lecture | Thursday, Nov. 4 |
Recurrent Neural Networks Long Short Term Memory (LSTM) |
[slides] DL book RNN chapter (optional) min-char-rnn, char-rnn, neuraltalk2 The Unreasonable Effectiveness of RNN (optional) Understanding LSTM Networks (optional) |
| Optional Discussion | Friday, Nov. 5 | Getting started with Project and Milestone Expectations | |
| Lecture | Tuesday, Nov. 9 | RNN's: continued | |
| Lecture | Thursday, Nov. 11 | No class | |
| Lecture | Tuesday, Nov. 16 | Exam Review | |
| Mid-Term | Thursday, Nov. 18 | Midterm to be held during regular lecture time | |
| Lecture | Tuesday, Nov. 23 | Do word embeddings, ElMO? Attention and Self-Attention in NLP Transformers |
[slides] [review sheet] |
| Lecture | Thursday, Nov. 25 | No class | |
| Lecture | Tuesday, Nov. 30 | Guest Lecture on Transformers by Andrew Drozdov |
[Slides] |
| Lecture | Thursday, Dec. 2 | TBD | |
| Lecture | Tuesday, Dec. 7 | Last lecture |