Azure Databricks and Spark SQL (Python)
What you'll learn
- Azure Databricks
- Data Lakehouse
- Delta Lake
- Spark SQL
- Big Data
- Real World Scenarios
- CI/CD on Databricks
- Source Control with Databricks Repos
- Basic SQL
- Basic Python
Databricks is one of the most in demand big data tools around. It is a fast, easy, and collaborative Spark based big data analytics service designed for data science, ML and data engineering workflows.
The course is packed with lectures, code-along videos and dedicated challenge sections. This should be more than enough to keep you engaged and learning! As an added bonus you will also have lifetime access to all the lectures… and I have provided detailed notebooks as a downloadable asset, the notebooks will contain step by step documentation with additional resources and links.
I have ensured that the delivery of the course is engaging and concise, the curriculum is extensive yet delivered in an efficient way. The course will provide you with hands-on training utilising a variety of different data sets.
The course is aimed at teaching you PySpark, Spark SQL in Python and the Databricks Lakehouse Architecture.
You will primarily be using Databricks on Microsoft Azure in addition to other services such as Azure Data Lake Storage Gen 2, Azure Repos and Azure DevOps.
The course will cover a variety of areas including:
Set Up and Overview
Azure Databricks Notebooks
Reading and Writing Data
Data Analysis and Transformation with Spark SQL in Python
Charts and Dashboards in Databricks Notebooks
Databricks Medallion Architecture
Accessing Data in Cloud Object Storage
Databases, Tables and Views in Databricks
Delta Lake / Databricks Lakehouse Architecture
Spark Structured Streaming
Delta Live Tables
Access Control Lists (ACLs)
Source Control with Databricks Repos
CI/CD on Databricks
Who this course is for:
- Anyone interested in working with Big Data and Spark
- Anyone interested in working with Databricks
- Anyone interested in working with cloud platforms
- Aspiring Azure Data Engineers
Malvik is an Instructor specialising in Data Analytics and BI.
He has spent 10+ years working for and consulting a number of publicly listed / unlisted companies as a Data and BI specialist. Over the course of his career Malvik has developed a skillset in data engineering, data analytics and business intelligence, with expertise across a number of technologies and programming languages including SQL, Python and R. He has worked extensively with a number of leading software platforms including Azure, Hadoop, Spark, Databricks, MongoDB, Oracle, MySQL, MS SQL Server, Qlik and more.