Nuts and bolts of MLFlow

MLflow offers an open source platform for the machine learning lifecycle, enabling training, reuse, and deployment of models with reproducible steps, through tracking, projects, models, and a central model registry.

Install and configure Visual Studio Code on Linux Ubuntu, set up a ml flow course working directory, enable the code command path, and launch VS Code for ongoing environment customization.

Verify your Python version is 3.6 or higher, install the Microsoft Python extension, create and activate a venv, and install ipykernel and pandas for Jupyter notebooks.

Activate your environment, install mlflow with pip, verify the installation via mlflow --version, and prepare to create your first runs in the next video.

Explore mlflow basics by activating your environment, opening a terminal, and using mlflow commands to manage artifacts and runs, with cloud options like SageMaker and Azure ML.

Create your first mlflow program by creating a file named Main Pie, importing mlflow, and printing Hello from ML Flow, then activate the environment and save before running.

Create your first MLFlow run, inspect run IDs, and log parameters using experiments and tags to organize nested runs for ML experiments.

Create experiments with mlflow.create_experiment by name and location, run inside the experiment using its ID, and list, fetch by name or ID, or delete experiments as needed.

Start an MLflow run inside a named experiment, then log parameters, metrics, text, figures, artifacts, dictionaries, and images to track experiment results.

Learn to create and log mlflow experiments with iris data, perform k-means clustering using scikit-learn, log shapes, inertia, and label distributions for reproducible results.

Learn to use mlflow to create an iris experiment, start runs, log data shapes, parameters, and artifacts, and perform k-means with optimal inertia.

Launch the MLflow user interface to explore experiments, runs, metrics, and artifacts, then compare multiple runs and visualize inertia and other metrics.

Explore the current architecture of mlflow and how running a script creates a run folder with artifacts, metrics, and related files stored locally.

Explore an architecture overview of mlflow flow, showing how it collects tracks, blocks, matrix params, and artifacts to the local file system while storing metrics and parameters in PostgreSQL.

Move mlflow artifacts, parameters, and metrics to a sqlite backend, configure the tracking uri, and explore the mlflow ui to view experiments and runs.

Explore mlflow architecture by introducing a NetFlow server that stores metrics, artifacts, and parameters locally or in a database, and demonstrate storage on a remote mlflow server.

MLflow architecture patterns detailing the backend store, artifact store, and model registry backend, with options for PostgreSQL, SQL Server, local filesystem, and cloud storage.

MLflow auto logging records metrics, parameters, and artifacts across frameworks like scikit-learn, enabling runs without manual log statements and easy visualization in the MLflow UI.

Explore the MLflow model registry to manage model lifecycle, versions, and metadata, and learn to configure a SQLite backend, register models, and track artifacts.

Explore managing model lifecycles in MLflow and consume production versions from the model registry using the MLflow client to load, predict, and validate the iris dataset.

Explore MLflow models and learn how to save models in MLflow format for cross-framework consumption. Inspect artifacts and flavors, and serve a Python flavor model via a rest endpoint.

Learn to serve MLFlow models via a REST API on localhost:5001, post JSON data with a data key, and interpret predictions (KMeans clusters) returned by the server.

Outline the MLflow architecture on AWS, provisioning an S3 artifact store, a school database, security groups, and an email flow server, with testing over port 5000 from a local machine.

Create the three buckets, including a history S3 bucket for artifacts and a backend store for metrics and experiments, detailing naming, region, block public access, and encryption.

Create an rds backend using aws relational database service with mysql, configure free tier settings, admin credentials, and a default vpc, and prepare ec2 deployment for the email flow.

Create an EC2 instance to host ml flow server, install engine x, configure it, start both services, and enable http access by updating the security group.

Configure an EC2 MLflow server, link to an S3 artifact store via a new IAM role and policy, and set up a MySQL backend on RDS with proper security.