Deploy Serverless Machine Learning Models to AWS Lambda

Use Serverless Framework for fast deployment of different ML models to scalable and cost-effective AWS Lambda service.

Created byMilan Pavlović

Last updated 12/2020

English

English [Auto],

What you'll learn

Deploy regression, NLP and computer vision machine learning models to scalable AWS Lambda environment
How to effectively prepare scikit-learn, spaCy and Keras / Tensorflow frameworks for deployment
How to use basics of AWS and Serverless Framework
How to monitor usage and secure access to deployed ML models and their APIs

Course content

8 sections • 62 lectures • 7h 45m total length

Introduction: what you will build during the course2:03
In the first lecture you will see a working web application whose backend side consists of various machine learning models deployed to AWS Lambda with the help of Serverless Framework. Whitin the next couple of course sections you will be able to build such systems too!
What is Serverless Computing ?2:30
In this lecture I describe main concepts of serverless computing. After completing the lecture, you will uderstand what does "serverless computing" mean in general.
What is AWS Lambda ?6:08
What is Serverless Framework ?1:50
Exposing ML Models through AWS Lambda2:53
Expose machine learning models through AWS Lambda and API Gateway by deploying with optional S3-stored models, detailing warm and cold starts and end-to-end prediction flow.
Basic Concepts from Introduction

Why Linux?1:59
In this lecture I will briefly explain why I decided to do the practical lectures on Linux.
Pre-configured Virtual Machine Download0:38
For Mac Users: Setup Instructions0:29
Installing VirtualBox2:36
Creating Ubuntu Virtual Machine6:57
Download and extract the Ubuntu VM image, install 7-zip, and create a new Witchel machine in VirtualBox. Allocate RAM and attach the VDI as the hard disk.
Initial Ubuntu Setup10:27
Configure Ubuntu in a virtual machine by adjusting CPU cores in VirtualBox, installing guest additions for full screen, updating software, and configuring keyboard language and time zone.
Installing Miniconda9:23
Install Miniconda to manage Python environments, download the Linux installer, run the bash script, accept the license, initialize conda in your terminal, and create a VM snapshot for safety.
Installing Visual Studio Code4:35
Install Visual Studio Code on Ubuntu using Ubuntu Software, launch it, and install the Python extension to enable linting, debugging, and IntelliSense for serverless deployment of Lambda functions.
Installing pip35:20
Install python3 and pip3 on Ubuntu via the terminal, download get-pip.py with wget, and run it with python3 to install pip. Then install test resources and create a backup snapshot.
What is Docker ?3:06
Discover Docker fundamentals, including containerization and the differences from virtual machines. Learn how Docker images package code and dependencies for portable, serverless deployment to AWS Lambda.
Installing Docker6:51
Installing Serverless Framework7:54
Configuring Serverless5:42

Serverless Create8:32
Create a hello world AWS lambda using the serverless framework, generating a python 3 boilerplate service and wiring event, context, and a proxy API gateway response.
Editing serverless.yml File11:34
First Deployment9:12
Set memory size to 128 megabytes and a 30-second timeout for the serverless hello world function, then deploy with serverless, complete packaging, and create the cloud formation stack and endpoint.
Supporting Services Overview11:45
Explore how API Gateway triggers AWS Lambda executions, deploy serverless machine learning models with the Serverless Framework, and monitor performance with CloudWatch and CloudFormation.

Intro to Dataset and Frontend Code7:10
In this lecture I will show you the dataset which we will use for our model training and explain how you will use frontend code.
Creating Virtual Environment with Conda7:07
In this lesson we will create a virtual Python environment with miniconda and install required packages.
Simple Dataset Exploration12:54
In this lecture we will use simple data description functions from pandas package.
Training the Model11:44
In this lecture we will train the SVR model in scikit-learn.
Saving the Model5:22
In this lecture we will discuss some model saving options.
Creating Project and Handler Prototype9:05
In this lecture we will create a serverless project and start editing handler.py file.
Developing Prediction Function15:02
In this lecture we will code a function which will serve as Lambda and provide predictions from trained machine learning model.
Testing Lambda Function Locally11:49
After this lecture you will be introduced with options for testing lambda functions locally.
Editing serverless.yml File11:47
In this lecture we will edit serverless.yml file.
Creating requirements.txt and Deploying Model9:29
After this lecture you will be introduced with requirements.txt file, why it is used and how you can invoke your deployed Lambda function.
Deploy your own scikit-learn model

Analyzing CloudWatch Reports6:21
After this lecture you will know what to pay attention on in CloudWatch logs.
Dealing With Cold Starts14:01
In this lecture I will show you a simple technique of keeping your Lambda functions warm.
Important Notice About Scaling3:13
In this lecture I'll give a more detailed overview about cold starts.
Basics of Usage Plans and API Keys14:27
After this lecture you will be able to create usage plans and API keys for your APIs.
Check S3 storage and Costs9:38
In this lecture I will give an overview of costs components for AWS services used so far.

Intro to spaCy NLP framework1:40
Explore deploying spaCy NLP models to AWS Lambda, including a named entity recognition model and a parts-of-speech tagging and parsing model, with nine-language support and visualizers.
Creating Virtual Environment with Conda6:43
Create a conda virtual environment named spacy mianus dev with python 3.6, install spacy with pip, download the small English model, and verify loading it in Python for lambda deployment.
spaCy Usage Example in Jupyter Notebook13:28
Creating Project with Serverless2:17
Create a serverless project using the serverless framework, generate a Python three boilerplate template, and install the serverless Python requirements plugin.
Coding Lambda Function9:47
Code an AWS Lambda function handler in Python to perform spaCy-based named entity recognition, loading the model outside the handler, returning spans as JSON in the response.
Unzipping Requirements in handler.py4:01
Updating handler.py1:49
Update handler.py to load the space model directly from the installed package instead of using spaCy loading, ensuring reliable deployment on AWS Lambda.
Editing serverless.yml File10:09
Adding requirements.txt and Local Testing4:30
Deployment and Global Testing10:57
Deploy serverless space function on AWS Lambda, packaging Python code and testing with CloudWatch for cold start timing. Explore endpoint testing via a web GUI and analyze named entity recognition.
Create PoS Tagging and Parsing Endpoint

Solution Architecture Overview1:52
In this lecture I’ll give you a brief overview of the architecture which you will build though the section to be able to deploy computer vision models such as ResNet50 or InceptionV3, to AWS Lambda.
Creating Virtual Environment with Conda4:51
In this lecture we will create a new conda environment for our keras example.
ResNet50 Usage Example in Jupyter Notebook14:34
In this lecture I will show you hot to download and use ResNet50 model with Keras framework.
Creating S3 Buckets5:08
In this lecture we will create two S3 buckets – first one for uploaded images, and the second for deep learning models.
Updated Usage Example17:04
Demonstrate accessing s3 buckets with boto3 in a notebook, download and load a model and an image, and run a prediction as preparation for deploying to aws lambda.
Creating Project and Editing Handler File10:19
In this lecture we will create project with serverless and start editing handler.py file.
Finishing Handler File12:32
In this video we will mainly finish editing handler.py file.
Updating Handler and Editing serverless.yml File9:15
In this lecture we will update our handler.py file and start editing serverless.yml file.
Finishing serverless.yml File9:57
In this lecture we will finish editing serverless.yml file.
Testing Lambda Function Locally6:36
In this lecture we will test whether our Lambda function will work locally.
Setting Up Requirements11:12
In this lecture we will setup all necessary Python packages required for Lambda to work.
Deploying and Global Testing4:01
In this lecture we will finally deploy our function to AWS Lambda service.
Image Upload Settings on AWS15:44
In this lecture we will setup our web page example to be able to upload an image to S3 bucket and obtain predictions from ResNet50 model.
Visualizing Predictions on the Web Page4:59
In this lecture we will test our web page example to obtain predictions from deployed ResNet50 model.
Deploy InceptionV3 Keras Model

Requirements

Created AWS Account
Basic familiarity with Python and Machine Learning
Basic undestanding of Linux and Terminal
Basic understanding of JavaScript and REST APIs, but not strictly required

Description

In this course you will discover a very scalable, cost-effective and quick way of deploying various machine learning models to production by using principles of serverless computing. Once when you deploy your trained ML model to the cloud, the service provider (AWS in this course) will take care of managing server infrastructure, automated scaling, monitoring, security updating and logging.

You will use free AWS resources which are enough for going through the entire course. If you spend them, which is very unlikely, you will pay only for what you use.

By following course lectures, you will learn about Amazon Web Services, especially Lambda, API Gateway, S3, CloudWatch and others. You will be introduced with various real-life use cases which deploy different kinds of machine learning models, such as NLP, deep learning computer vision or regression models. We will use different ML frameworks - scikit-learn, spaCy, Keras / Tensorflow - and show how to prepare them for AWS Lambda. You will also be introduced with easy-to-use and effective Serverless Framework which makes Lambda creation and deployment very easy.

Although this course doesn't focus much on techniques for training and fine-tuning machine learning models, there will be some examples of training the model in Jupyter Notebook and usage of pre-trained models.

Who this course is for:

Beginner Machine Learning and DevOps Engineers, Data Scientists or Solution Architects
All Data Scientists and ML practitioners who need to deploy their trained ML models to production, quickly and at scale, without much bothering with infrastructure

Deploy Serverless Machine Learning Models to AWS Lambda

What you'll learn

Explore related topics

Course content

Introduction5 lectures • 15min

Setting up your system13 lectures • 1hr 6min

Program Code and Solutions Availability1 lecture • 1min

Hello World from Lambda4 lectures • 41min

Deploying scikit-learn Regression Model10 lectures • 1hr 41min

Post Deployment Activities5 lectures • 48min

Deploying spaCy NLP Model10 lectures • 1hr 5min

Deploying Keras ResNet50 Model14 lectures • 2hr 8min

Requirements

Description

Who this course is for: