Apache Spark with Databricks
3.7 (233 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
925 students enrolled

Apache Spark with Databricks

Course to implement Big Data's Apache Spark on Databricks using a Microsoft's cloud service - Azure
3.7 (233 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
925 students enrolled
Created by Big Data Trunk
Last updated 9/2019
English [Auto]
Current price: $139.99 Original price: $199.99 Discount: 30% off
5 hours left at this price!
30-Day Money-Back Guarantee
This course includes
  • 3 hours on-demand video
  • 3 articles
  • 15 downloadable resources
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • This course will provide you an in depth knowledge of apache Spark and how to work with spark using Azure Databricks.
  • You will learn to Provision your own Databricks workspace using Azure cloud.
  • You will be able to create application on Azure Databricks after completing the course
  • You will be able to process continual streams of data with Spark streaming using azure event Hub
  • Transform structured data using SparkSQL and Dataframes
  • Build a binary classification application using the Mlib pipelines API.
Course content
Expand all 34 lectures 03:00:18
+ Overview
2 lectures 05:04

An overview of our course

Preview 02:30

Something about us … the BIG DATA TRUNK

Big Data trunk Introduction !!
+ Apache Spark Introduction
9 lectures 01:05:08

How we use to work with big Data before the introduction of Spark

Preview 11:42

How the Big data world drastically changed after the introduction of Spark

Big Data After Spark

An Introduction to APACHE SPARK

Introduction to Apache Spark

Introduction to Spark RDD

Spark - RDD
Lab - Ways for creating RDD
Quiz: Spark Introduction
4 questions

Brief Discussion about Spark Architecture

Spark Application Architecture

Types of Operations done on Spark RDD. We will mainly discuss about Transformation and actions. We will also discuss few sample codes to illustrate in a better way.

Spark Operations - Transformation & Action
Lab - Transformation & Action
DAG (Direct Acyclic Graph)
1 question
+ Databricks with Microsoft Azure
4 lectures 19:49

An introduction to Azure Databricks

Spark Databricks Analytics in Azure

Here in video we will briefly discuss why we need Azure Databricks even if we have Apache Spark

Why Databricks ?

We will provision Azure Databricks cluster and also we will learn to create a cluster in Azure Databricks

Provisioning Databricks workspace in Azure portal
Using Databricks community Edition
2 questions
+ Understanding Cluster and Notebooks in Databricks
1 lecture 03:18

We will learn how we can use notebook in databricks and how it can be associated to any Cluster in Databricks

Use notebook in Databricks
+ Working with Spark in Databricks
6 lectures 52:28

We will use Jupiter notebook to run queries on set of external data. Also we will learn to use SQL & Panda queries in notebook.

After that we will learn to plot different types of charts

Spark SQL
2 questions

We will Develop a word count program. Also we will run this program on set of external data by removing special characters from the dataset.

Developing Spark application
Removing Stopwords

In this video we will cover Databricks runtime Machine learning. We will use Binay Composition and decision tree model to train our Model

Developing Spark ML application

We will work on Streaming data from the twitter. We will use Azure eventHub for reading the Twits from Twitter.

Developing Spark Streaming applications (real time Twitter Data)

How the Spark can be optimized to perform more efficiently and effectively.

Optimizing Spark Performance
1 question
+ Spark Interview
10 lectures 33:29
Spark Core
Something more about RDD
Type Safety in Dataset
Justify Lazy Evaluation in Spark
Explain Fault Tolerance in Spark
Coalesce and Repartition
Spark Executor Memory
Garbage Collection Optimization in Spark
Learn More on Spark Interview
+ Bonus Section
2 lectures 01:01
Thanks You looking way forward !!
Bonus Lecture
  • Some Prior scripting knowledge, anyways we will be explainig all the codes line by line whichever we will be using in our labs
  • Free or paid subscription for Microsoft Azure portal.

In this course you will learn the basics of creating Spark jobs, loading data, and working with data. You’ll also get an introduction to running machine learning algorithms and working with streaming data. Databricks lets you start writing Spark queries instantly so you can focus on your data problems.

Azure Databricks accelerate big data analytics and artificial intelligence (AI) solutions, a fast, easy and collaborative Apache Spark–based analytics service.

Why Azure Databricks?

Productive : Launch your new Apache Spark environment in minutes.

Scalable : Globally scale your analytics and machine learning projects.

Trusted : Help protect your data and business with Azure AD integration, role-based controls and enterprise-grade SLAs.

Flexible : Build machine learning and AI solutions with your choice of language and deep learning frameworks.

We believe that when you learn something, you should be able to apply it somewhere. So, in this course, we are also providing you with some of the important spark interview questions , which will help you to crack the interview with flying colors.

Who this course is for:
  • Anyone who wants to learn Spark using Azure Databricks