A deep understanding of deep learning (with Python intro)
0% Completed
01 – Introduction
02 – Download all course materials
03 – Concepts in deep learning
Lesson 001 What is an artificial neural network
Video - 00:16:2 mins
Lesson 002 How models learn
Video - 00:12:26 mins
Lesson 003 The role of DL in science and knowledge
Video - 00:16:43 mins
Lesson 004 Running experiments to understand DL
Video - 00:13:3 mins
Lesson 005 Are artificial neurons like biological neurons
Video - 00:17:49 mins
04 – About the Python tutorial
05 – Math, numpy, PyTorch
Download Attachment
Reading
Lesson 002 Introduction to this section
Video - 00:2:6 mins
Lesson 003 Spectral theories in mathematics
Video - 00:9:16 mins
Lesson 004 Terms and datatypes in math and computers
Video - 00:7:5 mins
Lesson 005 Converting reality to numbers
Video - 00:6:33 mins
Lesson 006 Vector and matrix transpose
Video - 00:6:58 mins
Lesson 007 OMG it's the dot product!
Video - 00:9:45 mins
Lesson 008 Matrix multiplication
Video - 00:15:27 mins
Lesson 009 Softmax
Video - 00:19:26 mins
Lesson 010 Logarithms
Video - 00:8:26 mins
Lesson 011 Entropy and cross-entropy
Video - 00:18:18 mins
Lesson 012 Minmax and argminargmax
Video - 00:12:47 mins
Lesson 013 Mean and variance
Video - 00:15:34 mins
Lesson 014 Random sampling and sampling variability
Video - 00:11:18 mins
Lesson 015 Reproducible randomness via seeding
Video - 00:8:37 mins
Lesson 016 The t-test
Video - 00:13:58 mins
Lesson 017 Derivatives intuition and polynomials
Video - 00:16:39 mins
Lesson 018 Derivatives find minima
Video - 00:8:32 mins
Lesson 019 Derivatives product and chain rules
Video - 00:10:00 mins
06 – Gradient descent
Lesson 001 Overview of gradient descent
Video - 00:14:15 mins
Lesson 002 What about local minima
Video - 00:11:56 mins
Lesson 003 Gradient descent in 1D
Video - 00:17:11 mins
Lesson 004 CodeChallenge unfortunate starting value
Video - 00:11:30 mins
Lesson 005 Gradient descent in 2D
Video - 00:14:48 mins
Lesson 006 CodeChallenge 2D gradient ascent
Video - 00:5:16 mins
Lesson 007 Parametric experiments on g.d
Video - 00:18:56 mins
Lesson 008 CodeChallenge fixed vs. dynamic learning rate
Video - 00:15:33 mins
Lesson 009 Vanishing and exploding gradients
Video - 00:6:4 mins
Lesson 010 Tangent Notebook revision history
Video - 00:1:52 mins
07 – ANNs (Artificial Neural Networks)
Download Attachment
Reading
Lesson 001 The perceptron and ANN architecture
Video - 00:19:50 mins
Lesson 002 A geometric view of ANNs
Video - 00:13:38 mins
Lesson 003 ANN math part 1 (forward prop)
Video - 00:16:22 mins
Lesson 004 ANN math part 2 (errors, loss, cost)
Video - 00:10:54 mins
Lesson 005 ANN math part 3 (backprop)
Video - 00:12:10 mins
Lesson 006 ANN for regression
Video - 00:24:9 mins
Lesson 007 CodeChallenge manipulate regression slopes
Video - 00:18:58 mins
Lesson 008 ANN for classifying qwerties
Video - 00:22:22 mins
Lesson 009 Learning rates comparison
Video - 00:23:46 mins
Lesson 010 Multilayer ANN
Video - 00:19:51 mins
Lesson 011 Linear solutions to linear problems
Video - 00:8:14 mins
Lesson 012 Why multilayer linear models don't exist
Video - 00:6:20 mins
Lesson 013 Multi-output ANN (iris dataset)
Video - 00:26:59 mins
Lesson 014 CodeChallenge more qwerties!
Video - 00:11:56 mins
Lesson 015 Comparing the number of hidden units
Video - 00:10:00 mins
Lesson 016 Depth vs. breadth number of parameters
Video - 00:17:25 mins
Lesson 017 Defining models using sequential vs. class
Video - 00:13:17 mins
Lesson 018 Model depth vs. breadth
Video - 00:20:31 mins
Lesson 019 CodeChallenge convert sequential to class
Video - 00:6:37 mins
Lesson 021 Reflection Are DL models understandable yet
Video - 00:8:26 mins
08 – Overfitting and cross-validation
Lesson 001 What is overfitting and is it as bad as they say
Video - 00:12:29 mins
Lesson 002 Cross-validation
Video - 00:17:13 mins
Lesson 003 Generalization
Video - 00:6:9 mins
Lesson 004 Cross-validation -- manual separation
Video - 00:12:39 mins
Lesson 005 Cross-validation -- scikitlearn
Video - 00:21:1 mins
Lesson 006 Cross-validation -- DataLoader
Video - 00:20:27 mins
Lesson 007 Splitting data into train, devset, test
Video - 00:9:45 mins
Lesson 008 Cross-validation on regression
Video - 00:8:9 mins
09 – Regularization
Lesson 001 Regularization Concept and methods
Video - 00:13:39 mins
002 train() and eval() modes
Video - 00:7:14 mins
Lesson 003 Dropout regularization
Video - 00:21:56 mins
Lesson 004 Dropout regularization in practice
Video - 00:23:13 mins
Lesson 005 Dropout example 2
Video - 00:6:33 mins
Lesson 006 Weight regularization (L1L2) math
Video - 00:18:25 mins
Lesson 007 L2 regularization in practice
Video - 00:13:24 mins
Lesson 008 L1 regularization in practice
Video - 00:12:22 mins
Lesson 009 Training in mini-batches
Video - 00:11:32 mins
Lesson 010 Batch training in action
Video - 00:10:47 mins
Lesson 011 The importance of equal batch sizes
Video - 00:6:59 mins
Lesson 012 CodeChallenge Effects of mini-batch size
Video - 00:11:57 mins
10 – Metaparameters (activations, optimizers)
Lesson 001 What are metaparameters
Video - 00:5:2 mins
Lesson 002 The wine quality dataset
Video - 00:17:29 mins
Lesson 003 CodeChallenge Minibatch size in the wine dataset
Video - 00:15:38 mins
Lesson 004 Data normalization
Video - 00:13:12 mins
Lesson 005 The importance of data normalization
Video - 00:9:33 mins
Lesson 006 Batch normalization
Video - 00:13:16 mins
Lesson 007 Batch normalization in practice
Video - 00:7:38 mins
Lesson 008 CodeChallenge Batch-normalize the qwerties
Video - 00:5:7 mins
Lesson 009 Activation functions
Video - 00:17:59 mins
Lesson 010 Activation functions in PyTorch
Video - 00:12:12 mins
Lesson 011 Activation functions comparison
Video - 00:9:27 mins
Lesson 012 CodeChallenge Compare relu variants
Video - 00:7:49 mins
Lesson 013 CodeChallenge Predict sugar
Video - 00:17:6 mins
Lesson 014 Loss functions
Video - 00:16:47 mins
Lesson 015 Loss functions in PyTorch
Video - 00:18:42 mins
Lesson 016 More practice with multioutput ANNs
Video - 00:14:5 mins
Lesson 017 Optimizers (minibatch, momentum)
Video - 00:18:41 mins
Lesson 018 SGD with momentum
Video - 00:7:46 mins
Lesson 019 Optimizers (RMSprop, Adam)
Video - 00:15:40 mins
Lesson 020 Optimizers comparison
Video - 00:10:17 mins
Lesson 021 CodeChallenge Optimizers and... something
Video - 00:6:57 mins
Lesson 022 CodeChallenge Adam with L2 regularization
Video - 00:7:42 mins
Lesson 023 Learning rate decay
Video - 00:12:15 mins
Lesson 024 How to pick the right metaparameters
Video - 00:11:47 mins
11 – FFNs (Feed-Forward Networks)
Lesson 001 What are fully-connected and feedforward networks
Video - 00:4:58 mins
Lesson 002 The MNIST dataset
Video - 00:12:33 mins
Lesson 003 FFN to classify digits
Video - 00:22:20 mins
Lesson 004 CodeChallenge Binarized MNIST images
Video - 00:5:24 mins
Lesson 005 CodeChallenge Data normalization
Video - 00:16:16 mins
Lesson 006 Distributions of weights pre- and post-learning
Video - 00:14:48 mins
Lesson 007 CodeChallenge MNIST and breadth vs. depth
Video - 00:12:35 mins
Lesson 008 CodeChallenge Optimizers and MNIST
Video - 00:7:6 mins
Lesson 009 Scrambled MNIST
Video - 00:8:00 mins
Lesson 010 Shifted MNIST
Video - 00:11:25 mins
Lesson 011 CodeChallenge The mystery of the missing 7
Video - 00:10:47 mins
Lesson 012 Universal approximation theorem
Video - 00:8:31 mins
12 – More on data
Lesson 001 Anatomy of a torch dataset and dataloader
Video - 00:17:58 mins
Lesson 002 Data size and network size
Video - 00:16:35 mins
Lesson 003 CodeChallenge unbalanced data
Video - 00:20:5 mins
Lesson 004 What to do about unbalanced designs
Video - 00:7:45 mins
Lesson 005 Data oversampling in MNIST
Video - 00:16:30 mins
Lesson 006 Data noise augmentation (with devset+test)
Video - 00:13:16 mins
Lesson 007 Data feature augmentation
Video - 00:19:40 mins
Lesson 008 Getting data into colab
Video - 00:6:5 mins
Lesson 009 Save and load trained models
Video - 00:6:14 mins
Lesson 010 Save the best-performing model
Video - 00:15:18 mins
Lesson 011 Where to find online datasets
Video - 00:5:32 mins
13 – Measuring model performance
Lesson 001 Two perspectives of the world
Video - 00:7:1 mins
Lesson 002 Accuracy, precision, recall, F1
Video - 00:12:40 mins
Lesson 003 APRF in code
Video - 00:6:42 mins
Lesson 004 APRF example 1 wine quality
Video - 00:13:34 mins
Lesson 005 APRF example 2 MNIST
Video - 00:12:2 mins
Lesson 006 CodeChallenge MNIST with unequal groups
Video - 00:9:14 mins
Lesson 007 Computation time
Video - 00:9:56 mins
Lesson 008 Better performance in test than train
Video - 00:8:35 mins
14 – FFN milestone projects
Lesson 001 Project 1 A gratuitously complex adding machine
Video - 00:7:5 mins
Lesson 002 Project 1 My solution
Video - 00:11:18 mins
Lesson 003 Project 2 Predicting heart disease
Video - 00:7:15 mins
Lesson 004 Project 2 My solution
Video - 00:18:21 mins
Lesson 005 Project 3 FFN for missing data interpolation
Video - 00:9:35 mins
Lesson 006 Project 3 My solution
Video - 00:8:31 mins
15 – Weight inits and investigations
Lesson 001 Explanation of weight matrix sizes
Video - 00:11:54 mins
Lesson 002 A surprising demo of weight initializations
Video - 00:15:52 mins
Lesson 003 Theory Why and how to initialize weights
Video - 00:12:46 mins
Lesson 004 CodeChallenge Weight variance inits
Video - 00:13:14 mins
Lesson 005 Xavier and Kaiming initializations
Video - 00:15:42 mins
Lesson 006 CodeChallenge Xavier vs. Kaiming
Video - 00:16:55 mins
Lesson 007 CodeChallenge Identically random weights
Video - 00:12:41 mins
Lesson 008 Freezing weights during learning
Video - 00:12:58 mins
Lesson 009 Learning-related changes in weights
Video - 00:21:55 mins
Lesson 010 Use default inits or apply your own
Video - 00:4:36 mins
16 – Autoencoders
Lesson 001 What are autoencoders and what do they do
Video - 00:11:42 mins
Lesson 002 Denoising MNIST
Video - 00:15:48 mins
Lesson 003 CodeChallenge How many units
Video - 00:19:53 mins
Lesson 004 AEs for occlusion
Video - 00:17:55 mins
Lesson 005 The latent code of MNIST
Video - 00:21:57 mins
Lesson 006 Autoencoder with tied weights
Video - 00:24:14 mins
17 – Running models on a GPU
18 – Convolution and transformations
Lesson 001 Convolution concepts
Video - 00:21:34 mins
Lesson 002 Feature maps and convolution kernels
Video - 00:9:32 mins
Lesson 003 Convolution in code
Video - 00:21:5 mins
Lesson 004 Convolution parameters (stride, padding)
Video - 00:12:14 mins
Lesson 005 The Conv2 class in PyTorch
Video - 00:13:23 mins
Lesson 006 CodeChallenge Choose the parameters
Video - 00:7:10 mins
Lesson 007 Transpose convolution
Video - 00:13:41 mins
Lesson 008 Maxmean pooling
Video - 00:18:36 mins
Lesson 009 Pooling in PyTorch
Video - 00:13:43 mins
Lesson 010 To pool or to stride
Video - 00:9:47 mins
Lesson 011 Image transforms
Video - 00:16:57 mins
Lesson 012 Creating and using custom DataLoaders
Video - 00:19:6 mins
19 – Understand and design CNNs
Lesson 001 The canonical CNN architecture
Video - 00:10:47 mins
Lesson 002 CNN to classify MNIST digits
Video - 00:26:6 mins
Lesson 003 CNN on shifted MNIST
Video - 00:8:36 mins
Lesson 004 Classify Gaussian blurs
Video - 00:24:10 mins
Lesson 005 Examine feature map activations
Video - 00:27:50 mins
Lesson 006 CodeChallenge Softcode internal parameters
Video - 00:16:48 mins
Lesson 007 CodeChallenge How wide the FC
Video - 00:11:25 mins
Lesson 008 Do autoencoders clean Gaussians
Video - 00:17:10 mins
Lesson 009 CodeChallenge AEs and occluded Gaussians
Video - 00:9:37 mins
Lesson 010 CodeChallenge Custom loss functions
Video - 00:20:15 mins
Lesson 011 Discover the Gaussian parameters
Video - 00:16:59 mins
Lesson 012 The EMNIST dataset (letter recognition)
Video - 00:24:59 mins
Lesson 013 Dropout in CNNs
Video - 00:10:14 mins
Lesson 014 CodeChallenge How low can you go
Video - 00:6:45 mins
Lesson 015 CodeChallenge Varying number of channels
Video - 00:13:39 mins
Lesson 016 So many possibilities! How to create a CNN
Video - 00:4:42 mins
20 – CNN milestone projects
Lesson 001 Project 1 Import and classify CIFAR10
Video - 00:7:16 mins
Lesson 002 Project 1 My solution
Video - 00:12:2 mins
Lesson 003 Project 2 CIFAR-autoencoder
Video - 00:4:51 mins
Lesson 004 Project 3 FMNIST
Video - 00:3:52 mins
Lesson 005 Project 4 Psychometric functions in CNNs
Video - 00:11:55 mins
21 – Transfer learning
Lesson 001 Transfer learning What, why, and when
Video - 00:16:53 mins
Lesson 002 Transfer learning MNIST - FMNIST
Video - 00:10:6 mins
Lesson 003 CodeChallenge letters to numbers
Video - 00:14:25 mins
Lesson 004 Famous CNN architectures
Video - 00:6:46 mins
Lesson 005 Transfer learning with ResNet-18
Video - 00:16:44 mins
Lesson 006 CodeChallenge VGG-16
Video - 00:3:41 mins
Lesson 007 Pretraining with autoencoders
Video - 00:20:1 mins
Lesson 008 CIFAR10 with autoencoder-pretrained model
Video - 00:18:11 mins
22 – Style transfer
Lesson 001 What is style transfer and how does it work
Video - 00:4:36 mins
Lesson 002 The Gram matrix (feature activation covariance)
Video - 00:12:37 mins
Lesson 003 The style transfer algorithm
Video - 00:10:58 mins
Lesson 004 Transferring the screaming bathtub
Video - 00:22:16 mins
Lesson 005 CodeChallenge Style transfer with AlexNet
Video - 00:7:15 mins
23 – Generative adversarial networks
Lesson 001 GAN What, why, and how
Video - 00:17:23 mins
Lesson 002 Linear GAN with MNIST
Video - 00:21:55 mins
Lesson 003 CodeChallenge Linear GAN with FMNIST
Video - 00:9:50 mins
Lesson 004 CNN GAN with Gaussians
Video - 00:15:6 mins
Lesson 005 CodeChallenge Gaussians with fewer layers
Video - 00:6:5 mins
Lesson 006 CNN GAN with FMNIST
Video - 00:6:24 mins
Lesson 007 CodeChallenge CNN GAN with CIFAR
Video - 00:7:51 mins
24 – RNNs (Recurrent Neural Networks) (and GRULSTM)
Lesson 001 Leveraging sequences in deep learning
Video - 00:12:53 mins
Lesson 002 How RNNs work
Video - 00:15:14 mins
Lesson 003 The RNN class in PyTorch
Video - 00:17:45 mins
Lesson 004 Predicting alternating sequences
Video - 00:19:30 mins
Lesson 005 CodeChallenge sine wave extrapolation
Video - 00:24:49 mins
Lesson 006 More on RNNs Hidden states, embeddings
Video - 00:15:52 mins
Lesson 007 GRU and LSTM
Video - 00:23:8 mins
Lesson 008 The LSTM and GRU classes
Video - 00:13:26 mins
Lesson 009 Lorem ipsum
Video - 00:25:11 mins
25 – Ethics of deep learning
Lesson 001 Will AI save us or destroy us
Video - 00:9:40 mins
Lesson 002 Example case studies
Video - 00:6:39 mins
Lesson 003 Some other possible ethical scenarios
Video - 00:10:35 mins
Lesson 004 Will deep learning take our jobs
Video - 00:10:27 mins
Lesson 005 Accountability and making ethical AI
Video - 00:11:22 mins
26 – Where to go from here
27 – Python intro Data types
Lesson 001 How to learn from the Python tutorial
Video - 00:3:25 mins
Lesson 002 Variables
Video - 00:18:14 mins
Lesson 003 Math and printing
Video - 00:18:31 mins
Lesson 004 Lists (1 of 2)
Video - 00:13:31 mins
Lesson 005 Lists (2 of 2)
Video - 00:9:29 mins
Lesson 006 Tuples
Video - 00:7:40 mins
Lesson 007 Booleans
Video - 00:18:19 mins
Lesson 008 Dictionaries
Video - 00:11:51 mins
28 – Python intro Indexing, slicing
29 – Python intro Functions
Lesson 001 Inputs and outputs
Video - 00:7:1 mins
Lesson 002 Python libraries (numpy)
Video - 00:14:20 mins
Lesson 003 Python libraries (pandas)
Video - 00:13:57 mins
Lesson 004 Getting help on functions
Video - 00:7:36 mins
Lesson 005 Creating functions
Video - 00:20:27 mins
Lesson 006 Global and local variable scopes
Video - 00:13:20 mins
Lesson 007 Copies and referents of variables
Video - 00:5:45 mins
Lesson 008 Classes and object-oriented programming
Video - 00:18:46 mins
30 – Python intro Flow control
Lesson 001 If-else statements
Video - 00:15:3 mins
Lesson 002 If-else statements, part 2
Video - 00:16:58 mins
Lesson 003 For loops
Video - 00:17:37 mins
Lesson 004 Enumerate and zip
Video - 00:12:12 mins
Lesson 005 Continue
Video - 00:7:24 mins
Lesson 006 Initializing variables
Video - 00:18:1 mins
Lesson 007 Single-line loops (list comprehension)
Video - 00:15:25 mins
Lesson 008 while loops
Video - 00:19:30 mins
Lesson 009 Broadcasting in numpy
Video - 00:15:41 mins
Lesson 010 Function error checking and handling
Video - 00:17:42 mins
31 – Python intro Text and plots
Lesson 001 Printing and string interpolation
Video - 00:17:18 mins
Lesson 002 Plotting dots and lines
Video - 00:12:55 mins
Lesson 003 Subplot geometry
Video - 00:16:10 mins
Lesson 004 Making the graphs look nicer
Video - 00:18:48 mins
Lesson 005 Seaborn
Video - 00:11:8 mins
Lesson 006 Images
Video - 00:17:59 mins
Lesson 007 Export plots in low and high resolution
Video - 00:7:58 mins
32 – Bonus section
Download Attachment: Click here
Finish Course Early?
You have not completed all required lessons and assessments.
Your Progress
0%
