Python Programming: Build a Recommendation Engine in Django

Name: Python Programming: Build a Recommendation Engine in Django
Rating: 4.4 (16 reviews)

Collaborative Filtering with Python, Celery, Django, Worker Processes, Batch Predictions, SurpriseML, Keras, and more!

Created byJustin Mitchel

Last updated 2/2023

English

English [Auto],

What you'll learn

Learn how to integrate Django & Celery
Learn how to use HTMX with Django for Dynamic Loading (no JavaScript Needed)
Training a Machine Learning Model with SurpriseML and an example in Keras
Build a rating system in Django with dynamic rating buttons. These ratings can be used on any Django Model
Learn how to run periodic background task and/or schedule functions to run exactly when needed
How to perform batch inference effectively using Django for *any* large workloads and/or ML packages
How to load large datasets into a SQL database through Django Models
Where to find great datasets online
How to implement an "infinite" review page that will always give a new item after rating.
So much more!

Course content

11 sections • 43 lectures • 9h 35m total length

Welcome to Recommender2:45
Requirements & In-Depth Walkthrough15:34
Recommended Experience
Python 3.6+ (such as the 30 Days of Python course)
Django 3.2+ (such as the courses Your First Django Web Project or Try Django 3.2)
Celery with Django (such as Time & Tasks 2 course or the linked blog post in resources)
Where to get help2:00
Setup Project9:57
Django as a ML Pipeline Orchestration Tool2:58
Orchestrate machine learning pipelines with Django, collecting user data, preparing it with pandas, and training models via Celery with PyTorch, addressing cold start with fake data.

Generate Fake User Data6:38
Django Management Command to add Fake User Data11:27
Learn to build a Django management command within a profiles app to generate fake user data, using bulk create and a count argument to control output.
Our Collaborative Filtering Dataset7:50
Load The Movies Dataset into the Movie Django Model13:06
Load the movies dataset into the Django movie model by parsing a CSV with dict reader, validating release dates, and bulk creating records.
Create Ratings Model with Generic Foreign Keys13:29
Build a flexible Django ratings model using a generic foreign key and content types to rate multiple models, with reverse relations and admin-ready calculations of counts and averages.

Dynamic Requests with HTMX15:50
Learn to add dynamic content in Django with tmcs and Django HTML, enabling rating saves and list reordering without page reloads. Implement template switching and sorting (recent, unpopular) via get_queryset.
Rate Movies Dynamically with HTMX16:16
Infinite Rating Flow with Django & HTMX14:14
Develop an infinite rating flow in Django with HTMX by rendering a random movie, enabling skip and rate actions, and refreshing content via x-trigger headers and template swapping.
Rating Dataset Exports Model & Task23:33

Using Jupyter with Django7:50
Load Real Ratings to Fake Users15:29
Load real ratings into the Django database from a csv in a jupyter notebook, creating any missing users and transforming the data into rating records via pandas and bulk create.
Update Movie Data29:39
Identify and resolve inconsistencies between movie and rating data by locating missing movie IDs, enriching IMDB identifiers with pandas, and updating Django movie records and ratings accordingly.
Recommendations by Popularity16:27
Learn to build a Django recommendation engine by computing a popularity score from rating average and count, sorting by score to show popular and unpopular movies.

What is Collaborative Filtering13:14
Learn collaborative filtering for a django-based recommendation engine by using user ratings and latent features to predict scores and tailor suggestions, addressing cold-start.
Collaborative Filtering with Surprise ML9:50
Explore collaborative filtering with surprise ml by loading and cleaning data, setting a rating scale, cross-validating with rmse and mae, training the model, and exporting it with pickle for inference.
Surprise ML Utils & Celery Task For Surprise Model Training24:58
Batch User Prediction Task15:22

Storing Predictions in our Suggestion Model14:43
Store machine learning predictions in a Django suggestion model modeled after ratings, including did_rate and did_rate_timestamp, then bulk create, migrate, and verify did_rate updates after ratings.
Updating Batch Predictions Based on Previous Suggestions13:54
ML-Based Movies Recommendations View17:58
Trigger ML Predictions Per User Activity9:55
Position Ranking for Movie Querysets10:44
Explore how to implement position ranking for movie querysets in Django using window functions, dense_rank, and annotate to create a continuous position field for machine learning predictions.
Movie Embedding Idx Field and Task12:31
Movie Dataset Exports17:31
Schedule for ML Training, ML Inference, Movie IDX Updates, and Exports12:50

Requirements

Experience Python 3.6+ (such as the first 15/16 days from my course 30 Days of Python)
Django 3.2+ experience (such as my course Your First Django Web Project or any of the Try Django series)
Celery experience is a plus! (Such as my Time & Tasks 2 course)
Machine learning experience is a plus but not required (checkout my Hello World of Machine Learning Course)
Pandas basics is a plus but not required (checkout my Try Pandas Course)

Description

Build a recommendation engine using Django & a Machine Learning technique called Collaborative Filtering.

Users will rate movies and the system will automatically recommend new ones. These recommendations will be done in batches (ie not in real time) to unlock a more scalable system for training and helping thousands and thousands of users.

For this course, we'll use a real dataset called MovieLens; this dataset is downloaded in CSV and is used on all kinds of machine learning tutorials. What's special about this course is you'll load this dataset into a SQL database through a Django model. This alone might be worth watching the course as SQL databases are far more powerful than CSV files.

To do the batch inference we implement the incredibly powerful background worker process called Celery. If you haven't used Celery before, this will be an eye opening experience and when you couple it with Django you have a truly powerful worker process that can run tasks in the background, run tasks on a schedule, or a combination of both. Tasks in Celery are simply Python functions with a special decorator.

For rating movies, we'll be using HTMX. HTMX is a way to dynamically update content *without* reloading the page at all. I am sure you know the experience whenever you click "like" or "subscribe" , that's what HTMX gives us without the overhead of using 1 line of JavaScript. This course shows us a practical implementation of using HTMX not just for rating movies, but also sorting them, loading them, and doing much more.

The recommendation engine in Django is really a collection of 3 parts:

Web Process: Setup up Django to collect user's interest and provide recommendations once available.
Machine Learning Pipeline: Extract data from Django, transform it, and train a Collaborative Filtering model.
Worker Process: This is the glue. We'll use Celery to schedule/run the trained model predictions and update data for Django-related user recommendations.

Recommended Experience

Python 3.6+ (such as 30 Days of Python)
Django 3.2+ (such as Your First Django Web Project or Try Django 3.2)
Celery with Django (such as Time & Tasks 2 or this blog post)

Who this course is for:

Beyond the basics Django Developers (ie you completed a Try Django course)
Anyone interested in building powerful ML-heavy Web Applications
Anyone looking to learn about Python Celery for Worker processes
Anyone interested in building workflows that need to run along side of Django.

Python Programming: Build a Recommendation Engine in Django

What you'll learn

Explore related topics

Course content

Introduction5 lectures • 33min

Handling the Dataset5 lectures • 53min

Running Calculations with Django4 lectures • 53min

Python Celery2 lectures • 32min

Django Views & Auth3 lectures • 42min

User Ratings2 lectures • 26min

Dynamic Django with HTMX4 lectures • 1hr 10min

Jupyter Notebooks with Django4 lectures • 1hr 9min

Machine Learning & Collaborative Filtering4 lectures • 1hr 3min

Handling Predictions in Django8 lectures • 1hr 50min

Requirements

Description

Who this course is for: