Generative Deep Learning I

Learn how to encode images into a 2d latent space using a projection mapping e, and how to decode, sample, and interpolate within that latent space.

Build and train a PyTorch autoencoder with encoding and decoding to reconstruct grayscale images, using a mean square loss and Adam optimizer, and explore the latent space via interpolation.

Encode images into a two-dimensional latent and decode back to images using learned weights and biases. Train with loss minimization in PyTorch to ensure accurate reconstruction from latent space.

Train an mlp autoencoder on the mnist dataset to encode and decode handwritten digits, visualize the latent space with tsne, and interpolate latent points to generate intermediate reconstructions.

Learn to replace a multilayer perceptron with convolution layers to extract localized features and build a convolutional autoencoder that encodes the image into a latent space and decodes it back.

Implement a variational autoencoder in PyTorch by building a CNN autoencoder with mu and sigma, sampling z, and training with reconstruction and Kullback divergence.

Train a variational autoencoder on the CelebA dataset of 200k images with 14 attributes to generate new faces by sampling from a Gaussian latent space and interpolating between images.

Train a variational autoencoder on the Celeba face attribute dataset from Kaggle using PyTorch data loaders and a CNN encoder. Optimize with reconstruction loss and KL divergence using Adam.

Load a pre-trained variational autoencoder and evaluate reconstructions from the dataset. Interpolate in latent space, sample latents, and modify image attributes to generate varied outputs.

Learn to modify image attributes using a conditional variational autoencoder with conditional vectors in the decoder, enabling attribute changes like male to female without manual latent-space edits.

Explain the quantized variational autoencoder, replacing a global latent with a discretized per-pixel feature volume via a codebook, enabling part-wise image editing and reconstruction via an autoregressive pixelcnn.

Explore generative adversarial networks that map latent space to images with a generator and discriminator. Train them with alternating steps, using real data such as CelebA to guide learning.

Train a GAN in PyTorch by pairing a generator and discriminator in adversarial mode, transforming latent vectors to images from the celeb dataset, using cross-entropy loss and the Adam optimizer.

This lecture explains the limitations of GANs, such as mode collapse and unstable training, and introduces Wasserstein GAN with gradient penalty (WGAN-GP) to stabilize learning using a critic-based loss.

Implement a wgan-gp in pytorch by replacing the discriminator output with 1d vectors, removing sigmoid, and enforcing gradient penalty in training.

Train a CycleGAN to perform image translation on unpaired data using two generators, G and F, with cycling loss and discriminators to ensure realism.

Explore StyleGAN v2's weight modulation and demodulation, train in PyTorch, and improve style control, noise placement, and path length regularization for higher quality image generation.

Learn to implement StyleGAN v2's mapping network in PyTorch, mapping latent z to style w via eight equalized linear layers with z normalization and ReLU activations.

Explore how the generator block decodes images from a fixed initial constant using a two-stage style block, style modulation, and noise, via a modulation-demodulation weighted convolution to RGB output.

Train and test StyleGAN V2 with the lab LM library in Google Colab, using Celeba images and WGAN losses with gradient penalties, then generate images with pre-trained weights.

Learn to load and use a pretrained StyleGAN2-ADA model in PyTorch, generate realistic faces from latent vectors, and explore style mixing and interpolation across mapping and generator networks.

Project your image into the StyleGAN v2 latent and style spaces to edit faces, reversing the image, moving along a latent direction, and applying perceptual loss.

Learn to compute latent edit directions for labeled images by generating data, labeling with a classifier, and deriving svm boundaries in latent and style spaces for attribute manipulation.

Load the StyleGAN v2 model with the ffhq library, load and optimize the projected latent, then edit the image via the interface by adjusting age and smile.