AI Engineering with Modal

Name: AI Engineering with Modal
Rating: 5.0 (1 reviews)

A hands-on guide to building and deploying scalable AI systems with Modal.

Last updated 1/2026

English

What you'll learn

Build and deploy scalable AI infrastructure by defining custom container images, specifying GPU/CPU resources, and managing persistent data with Modal Volumes.
Develop a complete Automatic Speech Recognition (ASR) pipeline to transcribe long audio files in parallel using GPU-accelerated models on Modal.
Fine-tune a transformer encoder model for a text classification task on a custom dataset, leveraging Modal for GPU-powered training and experiment management.
Deploy trained machine learning models as scalable, live web APIs using Modal's built-in FastAPI endpoints for real-time inference.
Launch a high-throughput, OpenAI-compatible API for Large Language Model (LLM) inference using vLLM on Modal.
Build a High-Throughput Image Processing Pipeline and Build an Image Similarity Search Endpoint
Deploy an endpoint for Text2Image Generation with black-forest-labs/FLUX.1-Krea-dev and Qwen Image
Deploy an endpoint for Image Editing Using Qwen Image Edit
Setup complex GPU environments with Libraries like Flash Attention
Deploy remote MCP servers with custom tools and use them within MCP hosts like Claude Desktop

9 sections • 28 lectures • 6h 56m total length

Introduction to Modal4:38
In this lesson we we will setup our Modal account and environment.
Run Your First Modal Function11:23
In this lesson we will run our first Modal function.
Configuring Auto Scaling and Input Concurrency10:24
In this lesson we will see how you can configure the auto scaling and input concurrency behavior of your Modal functions.
Defining Images11:38
In this lesson we see how to define and build custom container images.
GPU Acceleration and Setting CPU/Memory Resources8:45
In this lesson we see how to utilize GPUs as well as set resource limits on CPUs and Memory.
Intro to Volumes9:51
In this lesson we see how persist data across function calls and containers by using volume objects.

Fine-Tuning Transformer Encoder Models for Classification19:29
In this lesson we talk a little about the ModernBERT encoder model and walk through some of the training code at a high level.
Fine-Tuning ModernBERT for Classification: Part 210:56
In this lesson we review different ways we can debug the training code on Modal containers interactively.
Fine-Tuning ModernBERT for Classification: Part 310:06
In this lesson we run the training code as well as look at the results in the Modal UI dashboard as well as the wandb dashboard.
Deploying the Trained Encoder Model to a Modal Web Endpoint23:33
In this lesson we learn how to deploy our trained Encoder model to a web endpoint.
Fine-Tuning ModernBERT on a Different Classification Dataset4:47
In this lesson we quickly show some results of fine-tuning ModernBERT on a different classification dataset, MAGE, to predict if text is generated by a human or an LLM.

Intro to Modal Sandboxes5:55
In this lesson we introduce Modal Sandboxes.
Building a Python Coding Sandbox9:30
In this lesson we look at our main class for interacting with the Python coding sandbox.
Digging More into the Coding Sandbox9:23
In this lesson we dig a little deeper into the code behind the coding sandbox.
Customizing the Coding Sandbox7:42
In this lesson we see how to customize the coding sandbox by defining any custom image and dependencies we want to use.
Building a Simple Agent with Coding Abilities24:42
In this lesson we will build a simple LLM agent that operates in a loop with function calling. For simplicity we will use the OpenAI API and function/tool calling. One of the functions we will give the LLM is the coding sandbox function we built in the previous lessons.
Making a Front End Web App for our Simple Agent with Coding Abilities15:08
In this lesson I show off the LLM Agent traces within a little web app I vibe coded in Cursor.

Image Embeddings with Siglip2 - Part 114:53
In this lesson we go over the idea of image embeddings and how to create them with Siglip2.
Image Embeddings with Siglip2 - Part29:30
In this lesson we build out the image processing scripts on Modal to scale to thousands of images.
Image Embeddings with Siglip2 - Part 317:07
In this lesson we build an endpoint for image similarity search by indexing our embeddings in FAISS.

Welcome to AI Engineering with Modal! In this course you will get a taste of some of things you can build with Modal.

I'm hoping this will inspire you to go and learn more on your own after doing these examples and videos.

In this course you will get to build:

A Scalable ASR Pipeline: Build a robust system to transcribe long audio files in parallel using a GPU-accelerated Automatic Speech Recognition model.
A Fine-Tuned Classification Model: Fine-tune a modern transformer model (ModernBERT) on a custom dataset for a text classification task and deploy it as a live API.
A High-Throughput LLM Endpoint: Launch an OpenAI-compatible API for a powerful Large Language Model (like Qwen or Gemma) using vLLM for blazingly fast inference.
Building a Coding Agent with Modal Sandboxes: Build your own code interpreter Agent with an LLM and an isolated secure cloud environment for executing arbitrary python code.
A High-Throughput Image Processing Pipeline: Build an Image Similarity Search Endpoint Application
Image Generation Endpoints with black-forest-labs/FLUX.1-Krea-dev , Qwen Image, and Qwen Image Edit
Setting up complex GPU environments with libraries such as flash-attention and axolotl
Deploy remote MCP servers with custom tools and use them in MCP hosts such as Claude Desktop

This course is for anyone familiar with Python and interested in AI.

Intermediate Python developers and software engineers looking to level up on building AI applications.