MLOps Masters

Learn why Linux is essential for MLOps, explore Linux fundamentals, set up a Linux environment on AWS EC2, and master essential Linux commands for production deployments.

Set up Linux on AWS EC2 by launching an Ubuntu instance, using WSL as an optional alternative, and connect to a production server terminal to execute commands.

Master essential Linux commands for MLOps, including file and directory operations (ls, mkdir, cd, rm, cp, mv), editing with vim, and managing permissions with chmod.

Master essential Linux commands for MLOps development using the Linux handbook notes. Open the book section to revise with examples and descriptions whenever you need.

Learn to use git and GitHub for version control, manage repositories and branches, and push and pull code to collaborate on team projects.

Create a new repository with the new button, choose public or private, add a readme and MIT license, then clone to your local machine with git bash.

Learn to manage code with git basics, including status checks, staging, commits, pushing to origin main on GitHub, pulling updates, and tracking commit IDs.

Create and switch to a new branch with Git checkout -b, commit changes, push to origin, and explore branches with Git branch, leaving the main branch unmodified.

Learn how data version control with DVC enables end-to-end ML pipelines and pipeline tracking in MLOps, from data ingestion to deployment and retraining.

Learn to build and track end-to-end ML pipelines with DVC, turning a Jupyter notebook into modular components for data ingestion, preprocessing, feature engineering, modeling, and evaluation.

This lecture shows why cloud is required for mlops and outlines end-to-end pipelines—from data ingestion to prediction—and highlights aws, gcp, and tools like SageMaker and Vertex AI.

Explore experiment tracking with MLflow, automating hyperparameter tuning beyond manual Excel tracking, and visualize results in a production-grade UI with a parallel coordinates graph.

Learn MLflow experiment tracking on DagsHub by building a wine quality regression project with elastic net, logging parameters and metrics, and comparing runs to select the best model.

Explore Docker-based containerization to build, share, and run ML applications with consistent results across environments, using images, containers, Docker engine, isolation, and Dockerfiles.

Learn how to install Docker on Linux and Mac with straightforward steps. Configure Docker Desktop on Windows using WSL, Ubuntu, and Windows features to run the Docker engine.

Learn to pull, run, and manage Docker images and containers, log in to Docker Hub, and containerize your app with a Dockerfile and push the image to the registry.

Learn to write a Docker file, build a custom image, and push it to Docker Hub while containerizing a Flask calculator app and running it on port 8080.

Explore building an end-to-end ml pipeline with advanced modular coding in Python, from problem statement to deployment, using class-based design, GitHub version control, and a modular training and prediction workflow.

Sign up for a GitHub account, create a repository named end to end, add a readme, gitignore (python), and MIT license; clone it and open in Visual Studio Code.

Create a reusable project template that automatically builds a full folder structure and files for an end-to-end machine learning project using a Python script and pathlib.

Set up a machine learning project by creating a virtual environment, installing requirements (pandas, notebook, numpy, scikit-learn, matplotlib, Flask), and configuring a local package with setup.py for src.

Implement a custom logging system and utility suite for an ML project, including a constructor for easy imports, YAML reading, config box access, and ensure typed exceptions.

Follow project workflow to update config YAML, schema YAML, and params YAML, configure components and pipeline, and launch the web endpoints with main.py and App.py, plus prepare an experiment notebook.

Learn to fetch the red wine quality dataset from Kaggle, load it with pandas, build an end-to-end pipeline with elastic net regression, and evaluate with RMSE and R2.

Learn to build a data ingestion pipeline in a Jupyter notebook, configure with config.yaml, and manage stage-specific artifacts to transition from notebook experiments to modular ML pipelines.

Convert notebook workflows into modular data ingestion components by copying from the notebook into config entities and a configuration manager, then implement a stage zero-one data ingestion pipeline.

Validate the dataset against a yaml schema, check all columns, and raise an error if missing, then train the model only on correct data.

Create and integrate a data validation component within the MLOps pipeline by importing data validation config, building the stage zero data validation, and wiring it into the endpoint with logs.

Update the data transformation config, read data, and perform a train test split to generate training and testing CSVs, while outlining modular, reusable components for feature engineering and EDA.

Update the entity and config, import the data transformation config, and implement stage three of the data transformation pipeline. Validate data status, run the pipeline, and prepare for model trainer.

Launches a model trainer in a Jupyter notebook, updates config.yaml and params.yaml, uses elastic net with alpha and L1 ratio, loads training data, and saves model as model.joblib or pickle.

Convert to modular ML code by defining the entity and configuration manager, creating a model_trainer component and pipeline, wiring an endpoint, and running tests successfully before model evaluation.

Create and run a model evaluation workflow by updating config, saving RMSE, MAE, and R2 scores to JSON, loading test data, predicting with the trained model, and saving results.

Create and integrate a model evaluation component into the ml pipeline, wire it to the endpoint, and prepare a prediction web app with dockerized deployment.

Create a simple prediction pipeline in prediction.py by loading the trained model from the artifact trainer and returning predictions, and build a user app with HTML/CSS and App.py deployment.

Build a Flask web API for a machine learning app by wiring routes for index and results, handling form inputs for wine quality prediction, and deploying with Docker and AWS.

Create a docker file using a python 3.8 slim buster image to dockerize the entire source code, copy into app, install requirements, and run app.py in linux.

Learn how CI/CD enables continuous integration and delivery for ML projects, implementing automated deployment with GitHub actions, Docker, ECR, and AWS EC2 to keep endpoints up during updates.

Learn to implement a complete ci/cd deployment using GitHub actions with AWS, Docker, and ECR: build and push images, deploy to EC2, and expose port 8080.

Learn to implement ci cd deployment with Jenkins and CircleCI, compare with GitHub Actions, deploy ML projects to AWS EC2 using Docker and ECR, with Jenkins pipeline and Docker Compose.

Learn to master continuous integration and deployment with CircleCI by building Docker images, pushing to AWS ECR, and deploying on EC2 via a self-hosted runner, using YAML config.

Build a modular ml pipeline for a chicken disease classification project, integrating DVC, MLflow, and ci/cd tooling like GitHub Actions, Jenkins, and CircleCI.

Set up a GitHub repository for end-to-end chicken disease classification project, including creating the repo, adding readme and Python .gitignore, license, cloning, and scaffolding a template in visual studio code.

Create a reusable project template by implementing template.py, arranging flexible folder structures, and configuring a CNN classifier workflow with Flask and GitHub CI/CD scaffolding.

Set up an end-to-end deep learning project with TensorFlow 2.12, a local cnn classifier via setup.py, a virtual environment, and install requirements.txt to enable data ingestion and Flask app scaffolding.

Explore a custom logging and utilities module for ML projects, including creating log files, importing and testing a logger, and using YAML with config box and robust exception handling.

Follow the project workflow to update config.yaml, params, and entity, then build data ingestion, model training, evaluation components, and stage-based pipeline with main.py and jupiter notebook.

Convert the data ingestion notebook into a modular pipeline by configuring config.yaml, downloading and unpacking chicken fecal images for binary classification.

Master modular data ingestion by copying notebook code into a config and component, wiring a stage zero-one pipeline, and executing via an endpoint with artifact management.

Prepare base model demonstrates using a pre-trained VGG16 from Keras for image classification, replacing the last dense layer for two classes, freezing layers, and saving the updated h5 model.

Create a modular base model component by updating the entity, config, and pipeline, then test via the main endpoint to ingest data and produce a base model artifact.

Develop a model trainer notebook to manage training with config yaml, parameters like epochs, batch size, augmentation, and image size, and implement Keras training and validation.

Build a modular model trainer component, wire in training configuration, and attach it to endpoint to run the pipeline, deliver artifacts and a trained model, paving the way for evaluation.

Evaluate multiple models with MLflow experiment tracking, compare image size, batch size, epoch, classes, weights, and learning rate, and select the best model using parallel coordinate plots.

Convert a notebook into a modular MLflow evaluation component, update entities and configuration, and run parameter-driven experiments to compare models and select the best performing one.

Demonstrates integrating DVC into a machine learning pipeline by defining DVC YAML with data ingestion, base model preparation, training, evaluation, and using DVC repro to track artifacts and dependencies.

Build a prediction pipeline using a Flask app, loading a trained TensorFlow model with load_model, preprocessing uploaded images, and predicting healthy versus coccidiosis-affected labels.

Build a flask based image uploader with a bootstrap front end, using base64 encoding, with train and predict routes, and docker deployment.

Create a docker file using Python 3.8 on a buster base that copies code, installs requirements, and runs the app. Deploy to AWS with Jenkins or CircleCI CI/CD tools.