Data and AI with Azure Databricks

Name: Data and AI with Azure Databricks
Rating: 4.4 (414 reviews)

Work with Apache Spark and Build Custom Machine Learning, Deep Learning, GenAI models and RAG Chatbots

Created byKuljot Singh Bakshi

Last updated 3/2026

English

What you'll learn

Data Analytics with Apache Spark in Azure Databricks
Machine Learning with Azure Databricks
Deep Learning with Azure Databricks
Generative AI with Azure Databricks
Implementation of complex concepts like RAG, Hyperparameter tuning, Model Serving with Azure Databricks

Course content

7 sections • 41 lectures • 8h 33m total length

Course Introduction3:06
Master data analytics on Azure Databricks using Apache Spark and Databricks SQL, then build and deploy machine learning, deep learning, and generative AI apps like retrieval augmented generation.

Understanding Data Warehouse, Data Lake and Data Lakehouse8:56
Understanding Spark: Introduction and Evolution from Hadoop10:51
Understanding Spark: Deep Dive and Usage14:54
Lab: Deploying an Azure Databricks Workspace (Hands-On Lab)9:56
Lab: Deploying a Managed Compute Instance (Hands-On Lab)6:42
Lab: Setting Up the GitHub Repo for the Labs (Hands-On Lab)3:17
Lab 1: Getting Comfortable with Spark (Hands-On Lab)19:44
Lab 2: Advanced Apache Spark Operations (Hands-On Lab)9:38
Lab 3: Delta Table Operations (Hands-On Lab)15:50
Introduction to the Medallion Architecture5:16
Lab 4: Implementing the Medallion Architecture with Spark (Hands-On Lab)14:49

Introduction to ML on Azure Databricks22:10
Understanding Basic ML Concepts13:28
Explore basic machine learning concepts, including features and target, training and test data, mean squared error, bias-variance trade-off, and cross-validation methods.
Understanding Linear Algorithms20:10
Explore linear algorithms, including linear regression and shrinkage methods like ridge and lasso, and learn logistic regression for binary classification, while mlflow on azure databricks logs hyperparameters to prevent overfitting.
Understanding non-Linear Algorithms15:49
Explore non-linear machine learning with decision trees and ensembles, learning how data splits, information gain, Gini impurity, pruning, bagging, random forest, and boosting improve accuracy and control overfitting.
Lab: Training our Diabetes Predictor (Hands-On Lab)28:49
Lab: MLflow for Experiments and Model Logging (Hands-On Lab)13:53
Lab: Deploying Model to a REST API Endpoint (Hands-On Lab)10:14
Lab: Hyperparamter tuning using Optuna (Hands-On Lab)11:45
Introduction to Deep Learning Models15:55
Explore how deep learning uses neural networks with weights, biases, and activation functions to learn through loss and backpropagation, powering models on Azure Databricks.
Lab: Deep Learning (Hands-On Lab)21:01

Lab: Deploying a Foundational LLM and AI Playground (Hands-On Lab)13:31
Lab: Querying LLM with the Databricks SDK (Hands-On Lab)9:30
Lab: Querying LLM with the OpenAI SDK (Hands-On Lab)9:22
Introduction to MLflow and PyFunc for Custom AI Models and Engines8:00
Introduction to the Mosaic AI Framework in Databricks8:14
Lab: Registering a LLM Application using MLflow (Hands-On Lab)20:15
Lab: Deploying LLM Application as Mosaic AI Endpoint (Hands-On Lab)6:40

Requirements

Familiarity with Python Programming Language is required
A mind open to ideas and new perspectives

Description

Unlock the transformative potential of Azure Databricks with this comprehensive course on Data Analytics, Machine Learning, Deep Learning, and Generative AI. Designed for professionals and enthusiasts, this course equips you with cutting-edge skills to harness the power of Apache Spark and seamlessly integrate it with Azure Databricks for data-driven solutions.

Azure Databricks, a unified analytics platform, is revolutionizing how businesses analyze data and deploy machine learning at scale. This course dives deep into its robust features, helping you process large datasets with Apache Spark and extract meaningful insights in real time.

You will explore the full spectrum of machine learning and deep learning applications, from predictive modeling to advanced neural networks. Learn to train, optimize, and deploy models effectively while mastering Databricks’ MLflow for seamless model management and tracking.

But we don’t stop there! The course also delves into Generative AI, the forefront of AI innovation. Understand how to leverage generative models to create text, images, and synthetic data for diverse use cases. From foundational concepts to advanced implementations, you'll learn to build intelligent systems that think, learn, and generate.

Key highlights of the course:

Comprehensive introduction to Azure Databricks and its architecture.
Real-time data analytics with Apache Spark.
Building scalable machine learning pipelines.
Deep learning fundamentals and applications in Azure.
Harnessing Generative AI for innovative solutions.
Hands-on projects and real-world scenarios for practical understanding.

Whether you're a data engineer, machine learning practitioner, or an AI enthusiast, this course is your gateway to mastering modern data science and artificial intelligence with Azure Databricks. Join now and elevate your career to the next level!

Who this course is for:

Software Engineers
Data Scientists
Data Analysts
Machine Learning Engineers
AI Enthusiasts
Cloud Architects
Generative AI Enthusiasts

Data and AI with Azure Databricks

What you'll learn

Explore related topics

Course content

Introduction1 lecture • 3min

[NEW] Important Concepts4 lectures • 34min

[NEW] Data Analytics using Apache Spark and SQL11 lectures • 2hr

[NEW] Machine Learning and Deep Learning using Azure Databricks10 lectures • 2hr 53min

[NEW] Generative AI Primer and Fundamentals3 lectures • 19min

[NEW] GenAI And Databricks - Match made in Heaven!7 lectures • 1hr 16min

[NEW] RAG - Using your Enterprise Data5 lectures • 1hr 29min

Requirements

Description

Who this course is for: