Introduction to Diffusion Models

Apply the forward diffusion process to gradually add noise to data, forming a Gaussian distribution, then train neural networks to reverse the process and recover the original signal.

Implement the forward diffusion process for diffusion models in Python, using NumPy, Matplotlib, and scikit-learn to sample 2D data and add time-step noise with beta schedules.

Derive diffusion equations to express x_t via x_0 and a single gaussian epsilon, and rewrite as x_t = sqrt(alpha_bar_t) x_0 + sqrt(1 - alpha_bar_t) epsilon for efficient GPU implementation.

This lecture demonstrates implementing diffusion models in PyTorch by precomputing betas and alpha terms, avoiding loops, and deriving the forward process over time steps with mu, sigma, and epsilon.

Explore diffusion process visualization by building a PyTorch-based pipeline that batches data, converts numpy arrays to tensors, and plots forward and reverse steps to match the paper's figure.

Explore the reverse diffusion process in diffusion models, starting from a Gaussian distribution and denoising toward data using neural networks to estimate the mean and covariance through training.

Explore the reverse diffusion process by computing x_{t-1} from x_t using a time-conditioned model to predict the mean and covariance, with a plan to implement an MLP.

Explore time-dependent diffusion model architectures, including time concatenation and amortized shared bases with per-time-step heads; implement a two-layer MLP in PyTorch to predict the mean and diagonal sigma.

Demonstrates implementing and testing the diffusion model's reverse process for sampling, starting from noise, iterating over time steps, and visualizing results.

Visualize the diffusion model's reverse process by sampling 3000 points and comparing forward and backward trajectories side by side, reproducing a paper figure with red samples at key time steps.

Maximize the model log likelihood of data by optimizing a tractable evidence lower bound (ELBO) using Jensen’s inequality, Monte Carlo estimation, and Gaussian KL divergence between forward and reverse paths.

Explore how equation 12 leads to equation 13 in diffusion models, using entropy, conditional entropy, and KL divergence, and relate the reverse trajectory to Bayes and Gaussian assumptions.

Implement diffusion model training equations by performing Monte Carlo integration over random time steps, estimating KL divergence between q and p, and iterating mu and sigma adjustments.

Implement diffusion model training equations, compute sigma_q and mu_q, start at t=2, derive x_t from x_0 and x_T, and prepare a PyTorch training loop with loss for GPU acceleration.

Implement a self-contained training loop that samples batch data, computes loss with the model, zeroes gradients, backpropagates, and updates weights with an optimizer, while logging loss and plotting progress.

Train diffusion models on GPU using platforms like Google Colab or Paperspace, set up persistent notebook environments, run with PyTorch and Stable Diffusion, and manage device placement and plotting.

Correct typos in the diffusion model equations, include self dot t in the standard deviation's square root, and apply alpha bar t minus one, then compare epoch results.

Reproduce the paper’s figure by training a diffusion model across epochs, observing density evolve from a Gaussian distribution toward the paper’s results, with results varying by run.

Explore diffusion models, from forward diffusion and reverse process to variational training, with fixed betas and a reparameterized mean to boost sampling quality.

Explore incorporating time dependence in diffusion models by feeding time as an encoded input with transformer positional encoding and self-attention, aiming to reproduce MNIST results.

Implement the diffusion model training loop by sampling batches, times, and noise. Apply the sampling loop via the reverse process, sampling from gaussian noise, and the reparameterization trick with beta.

Explore implementing a diffusion U-Net in PyTorch, converting from TensorFlow, and building a time-step embedding for training. Practice includes downsampling, upsampling, attention, and remaining blocks to implement.

Explore implementing a U-Net downsampling block for diffusion models, including time embedding, convolutional downsampling, and preparing for the middle and upsampling stages in PyTorch.

Learn to implement a U-Net upsampling layer in PyTorch by upsampling with nearest neighbor, followed by a 2D convolution, ensuring the output matches the input after upsampling.

Explore the u-net diffusion architecture, focusing on Resnet blocks, attention, and downsampling/upsampling to adjust spatial resolution. Implement the Resnet module with a get timestep embedding.

Implement a ResNet block for a U-Net diffusion model by combining group normalization, convolutions, time-step embeddings via an MLP, dropout, and a skip-connected residual path, preparing for an attention block.

Fix a tensor shape mismatch in a U-Net ResNet block for diffusion models by adjusting H and W broadcasting across numpy, TensorFlow, and PyTorch, and validate upsampling.

Learn to implement a U-net attention block for diffusion models, including q, k, v projections, group normalization, reshaping, softmax, and a forward pass with channel handling.

Implement a U-Net attention mechanism for diffusion models and validate the attention block on CPU and GPU. Assemble the building blocks into a unit with 35.7 million parameters for CIFAR.

Finish the U-Net architecture by integrating time embeddings, two MLPs, and a full forward pass, then implement downsampling, middle, and upsampling blocks with resnet and attention blocks.

Finish the U-Net implementation for a diffusion model by wiring inputs through convolution and down blocks with time embeddings, managing skip connections and preparing the middle and upsampling stages.

Finish the U-Net by implementing a middle resnet and attention block, then add an upsampling block to restore the input shape, and prepare the module list for next session.

Finish the u-net implementation for a diffusion model with 35 million parameters, detailing upsampling, resnet blocks, attention blocks, time embeddings, and the required concatenations from downsampling paths.

Finish the unet implementation by correcting spatial sizes with upsampling and attention to ensure 32 by 32 outputs and 128 channels, and verify 35.7 million parameters for the diffusion model.

Implement a denoising diffusion probabilistic model in PyTorch by building a modular unit, upsampling, and attention blocks, then develop training and sampling routines.

Implement training for denoising diffusion probabilistic models by converting pseudocode to code: sample x0, sample t, compute epsilon, predict epsilon, minimize mse, and perform gradient updates.

Explore denoising diffusion probabilistic models and sampling, detailing training-loop loss handling, gaussian noise sampling, and the role of alpha, alpha bar, beta, and mean-variance updates.

Learn to add a tqdm-based progress bar to sampling in denoising diffusion probabilistic models, with a default on/off toggle, and implement a sample batch workflow using MNIST data.

Implement and troubleshoot the training loop for denoising diffusion probabilistic models, configuring batch size of 64, time steps, and Adam optimizer to monitor training loss and visualize progress on CPU.

Visualize denoising diffusion probabilistic model sampling to generate 81 samples in a 9x9 grid, tune tqdm usage, incorporate timestep embeddings, and save epoch plots and models every 5000 epochs.

Train denoising diffusion probabilistic models on the gpu, set up the training loop, ensure device consistency, and monitor loss and samples to accelerate and validate diffusion model training.

Log results every 5000 epochs and save on the last epoch; observe the decreasing training loss, the mean squared error between the means, and Mnist data before moving to inpainting.

Explore how diffusion models trained for image generation enable inpainting by combining forward and reverse processes, sampling, and masking to preserve context while reconstructing missing regions.

Explore implementing inpainting with diffusion models by combining forward process samples and reverse process generation, guided by a mask to preserve original regions while inpainting others.

Fixes a bug in diffusion model inpainting by correcting the sampling mean from X to X times sqrt(1 - BT), improving inpainting results on partial images.

Visualize diffusion model animations by tracing the reverse process from noise to image, showing sequential evolution over time steps.

Explore how to visualize diffusion model progress by clipping outputs, implementing a beta scheduler, and animating image evolution across selected time steps.

Explore animations for diffusion models by designing exponential-like time steps, managing indices, deduplicating, clipping, and reversing sequences, then learn to use stable diffusion via API and code edits.

Showcases stable diffusion, transporting the diffusion process to latent space via an encoder–decoder and enabling text and bounding-box conditioning of image generation with open-source code.

Explore how to use stable diffusion via the Hugging Face API and diffuser library to generate text-to-image results, perform image-to-image and inpainting, and leverage version 2.1 models.

Tune diffusion model parameters such as inference steps and guidance scale to trade image quality for speed. Explore negative prompts and reproducibility via seeds and latent inputs.

Learn how to use a generator parameter in the Hugging Face Stable Diffusion API to seed and reproduce images by setting and restoring generator state, manual seeds, and per-image seeds.

Explore the stable diffusion open-source codebase, compare it to our scratch implementations, and learn how text-to-image generation uses a latent diffusion model with prompt conditioning and safety checks.

Learn a simple inpainting workflow using stable diffusion v1, load image and mask pairs, encode to latent space, and sample to reconstruct images.

Explore how a 300-line image streaming script uses latent diffusion to transform an input image into a new image via a prompt, encoding to latent space and decoding.

Explore denoising diffusion implicit models, a non-markovian generalization of ddpm that enables 10–50x faster sampling, model reuse, and latent space interpolation with a consistent training objective.