Udemy
    •  
    •  
    •  
    •  
    •  
    •  
    •  
    •  
Subscribe
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
Mastering Generative Vision & Video: From GAN to Flow to DiT
New
20 students
Created byVinit Singh
Last updated 5/2026
English

What you'll learn

  • Build VAEs, GANs and Vision Transformers from scratch, understanding reparameterisation, minimax training and patch embeddings that underpin Stable Diffusion
  • Implement DDPM, Latent Diffusion Models and Flow Matching, understanding ODE solvers and time-step formulations used in production systems like SD 3.5 and Flux
  • Control and accelerate image generation using ControlNet, IP-Adapters, Consistency Models and adversarial distillation techniques like SDXL Turbo & Flux Schnell
  • Build spatiotemporal video generation systems using Diffusion Transformers, temporal attention and optical flow, with reference to Sora, Veo 2 and Gen-3.

Course content

5 sections • 316 lectures • 31h 39m total length

  • Introduction to Module 04:03
  • 0.1 Autoencoders (AEs) and VAEs4:41
  • 0111 Autoencoders (AEs) vs. Variational Autoencoder10:08
  • 0112 Ambient Space (The Chaos), Manifold,Latent Space5:41
  • 0113 The Core of Gen AI: Manifold, Latent Space & Autoencoders6:05
  • 0114 The Decoder - From Blueprint Back to Image5:27
  • Code Eg 0.1 part 1 Explaination3:47
  • 0121 Autoencoder (AE) Architecture Overview2:11
  • 0122 Autoencoder Implementation & Limitations4:37
  • 0123 Autoencoder Loss function6:44
  • Code Eg 0.1 part 2 Explaination4:40
  • 0131 Variational Auto-encoder (VAE) Architecture Overview2:25
  • 0132 Variational Autoencoders The Probabilistic Revolution4:05
  • 0133 The Reparameterization Trick in Generative Computer Vision (VAE)5:52
  • 0134 Understanding VAE Loss & The AE vs VAE Showdown5:02
  • Code Eg 0.1 part 3 Explaination6:42
  • 0141 AE vs. VAE Comparison6:53
  • 0142 β-VAE Disentangled Representations5:21
  • 0143 Conditional VAE (cVAE)6:04
  • 0144 Real-World Applications of AEs & VAEs4:05
  • Code Eg 0.1 part 4 Explaination4:51
  • 0.2 Generative Adversarial Networks (GANs) - Learning Objectives5:43
  • 0211 Generative Adversarial Networks (GANs)10:28
  • 0212 GAN Training Loop Path to Nash Equilibrium6:24
  • 0213 The Mathematical Engine - The Minimax Game5:40
  • Code Eg 0.2 part 1 Explaination5:41
  • 0221 DCGAN Architecture The Blueprint for Stable GANs11:10
  • 0222 GAN Training Choreography 1 Discriminator Training5:06
  • 0223 GAN Training Choreography Part 2 Generator Training3:23
  • 0224 The Complete Choreography3:06
  • Code Eg 0.2 part 2 Explaination7:56
  • 0231 GAN Challenges - Mode Collapse & Vanishing Gradients3:54
  • 0232 The Solution - Wasserstein GAN (WGAN)5:33
  • 0233 Advantages of WGAN & Implementation Details2:16
  • Code Eg 0.2 part 3 Explaination6:27
  • 0241 StyleGAN Architecture The Disentangled Synthesis5:34
  • 0242 Advanced GAN Architectures StyleGAN4:10
  • 0243 CycleGAN Unpaired Image to Image Translation2:21
  • 0244 CycleGAN Translation Without Paired Data5:17
  • 0245 Evaluating the Unsupervised GAN Metrics6:58
  • Code Eg 0.2 part 4 Explaination5:13
  • 0.3 Introduction to Vision Transformers (ViT) - Learning Objectives5:43
  • 0311 Vision Transformers (ViT) - The Paradigm Shift6:14
  • 0312 The Vision Transformer ( ViT) Architecture7:58
  • 0313 Why Vision Needs a Global View8:18
  • 0314 CNN vs. ViT A Tale of Two Perspectives3:22
  • Code Eg 0.3 part 1 Explaination4:43
  • 0321 The Engine of Vision - Self-Attention Explained4:09
  • 0322 Self-Attention & Multi-head attention6:44
  • 0323 From Pixels to Tokens - The ViT Patching Process4:05
  • 0324 Step 1 - Flattening Patches to Token Embeddings5:49
  • 0325 The Complete Pipeline3:23
  • 0326 Step -2 Linear Projection From Pixels to Visual Words5:02
  • Code Eg 0.3 part 2 Explaination8:34
  • 0331 Step 3 - Positional Encoding Vision Spatial Memor9:01
  • 0332 Step 4 - Transformer Encoder The Processing Core8:00
  • 0333 Layer Normalisation, MLP & Residual Connections7:34
  • Code Eg 0.3 part 3 Explaination6:36
  • 0341 Energy Landscape-Aware ViT (ELA-ViT)3:11
  • 0342 Hypergraph Vision Transformer (HgVT)2:47
  • 0343 2025 Vision Transformer Innovations4:09
  • 0344 Data-efficient Image Transformer ( DeiT)9:47
  • 0345 Swin Transformer Hierarchial Vision Transformer8:02
  • Code Eg 0.3 part 4 Explaination3:20

Requirements

  • Essential Technical Knowledge: Completion of a foundational computer vision course covering CNNs, image classification, and basic deep learning — or equivalent practical experience. This course is explicitly designed as a continuation of the "Mastering Computer Vision: From Pixel to Detection to Gen-CV" course and assumes that level of preparation. Solid Python programming skills including comfort with object-oriented programming, working with libraries, and writing training loops from scratch. Working knowledge of PyTorch or TensorFlow — students should be able to define a model, write a training loop, load data, and run inference without step-by-step guidance. Basic understanding of neural network fundamentals — forward pass, backpropagation, loss functions, gradient descent, and activation functions. Familiarity with Convolutional Neural Networks and how they process image data — feature maps, pooling, and spatial hierarchies.
  • Recommended but Not Strictly Required: Prior exposure to attention mechanisms and the Transformer architecture is helpful, as Module 0 covers Vision Transformers at an accelerated pace assuming some prior familiarity. Basic understanding of probability and statistics — particularly concepts like distributions, sampling, and KL divergence — will help with the diffusion and VAE modules. Familiarity with Jupyter Notebooks and running experiments on cloud GPU environments (Google Colab, Kaggle, or similar).
  • Hardware and Software: A computer capable of running Python 3.8 or higher with standard deep learning libraries installed. Access to a GPU environment for running labs — cloud GPU platforms are acceptable and recommended for students without local GPU resources. No specialized hardware is required beyond access to a free-tier cloud GPU for practical lab sessions.
  • This course is NOT suitable for: Complete beginners to deep learning or Python programming. Students with no prior exposure to convolutional neural networks or image classification. Those looking for a no-code or prompt-engineering course — this is an implementation and architecture-focused engineering course.

Description

Mastering Generative Vision and Video: From GAN to Flow to DiT

The Complete Engineering Guide to Modern Generative AI — Images, Video, and Audio-Visual Synthesis

Generative AI is no longer a research curiosity. It is the engine behind billion-dollar products, production pipelines at studios and startups, and the most sought-after engineering skillset in the AI job market today. Stable Diffusion, Sora, DALL-E, Runway, Midjourney, Kling, and Veo — every one of these systems is built on the architectural foundations this course teaches from first principles to production implementation.

This course picks up exactly where classical computer vision ends. You already understand CNNs, segmentation, and detection. Now it is time to master the generative side — the models that do not just recognize the visual world, but create, transform, and synthesize it.

"Mastering Generative Vision and Video: From GAN to Flow to DiT" is the only course that takes you through the complete evolution of generative architectures in a single, coherent learning journey. You will start with the foundational building blocks — Variational Autoencoders, GANs, and Vision Transformers — and progressively advance through Latent Diffusion Models, Flow Matching, ControlNet, Consistency Models, and finally Diffusion Transformers (DiT), the architecture powering Sora and the next generation of video generation systems.

The curriculum is structured around five modules covering 19 lectures of hands-on, implementation-focused content.


Module 0 ensures every student has the right foundation with VAEs, GANs, and ViT before entering the diffusion world.

Module 1 takes you from DDPM probability theory all the way to Flow Matching and ODE solvers.

Module 2 dives deep into control and acceleration — ControlNet, IP-Adapters, LCM Distillation, SDXL Turbo, and Flux Schnell.

Module 3 introduces spatiotemporal generation for video, covering DiT-based architectures, Sora, Veo 2, temporal attention, optical flow, and frame interpolation.

Module 4 closes the loop with generative audio-visual synchronization — neural audio synthesis with AudioLM and MusicGen, unified AV generation with Veo, lip-sync architectures with Wav2Lip, and latent audio-video alignment metrics.

This is not a course about prompting or using AI tools. This is an engineering course. You will understand the mathematics, implement the architectures, and build systems capable of generating images, videos, and synchronized audio-visual content.

Whether you are an AI engineer wanting to work on foundation model teams, a researcher building the next generation of generative systems, a developer integrating generative capabilities into production pipelines, or a technical entrepreneur building a generative AI product, this course gives you the complete, rigorous, and practical foundation to do it.

The demand for engineers who understand these systems at an architectural level is growing faster than the supply. This course is your path to becoming one of them.

Who this course is for:

  • 1. AI engineers and developers who want to move beyond recognition tasks and build generative image, video, and audio-visual systems using diffusion models, flow matching, and transformer architectures.
  • 2. Students who have completed a foundational computer vision course and are ready to advance into generative AI, learning the architectures behind Stable Diffusion, Sora, ControlNet, and Veo.
  • 3. Machine learning researchers and practitioners who want hands-on implementation experience with state-of-the-art generative models including DiT, LCM, SDXL, Flux, and audio-visual synthesis systems.
  • 4. Software developers and technical entrepreneurs building generative AI products who need architectural understanding beyond prompt engineering to integrate and customize foundation models.
  • 5. Data scientists and deep learning engineers looking to specialize in generative vision and video, one of the fastest growing and highest paying areas in the current AI job market.

Report abuse