Development
Development Tools
Apache Spark
Apache Spark with Databricks
3.7 (233 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
925 students enrolled

Apache Spark with Databricks

Course to implement Big Data's Apache Spark on Databricks using a Microsoft's cloud service - Azure
3.7 (233 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
925 students enrolled
Created by Big Data Trunk
Last updated 9/2019
English
English [Auto]
Preview this course
Apache Spark with Databricks
Current price: $139.99 Original price: $199.99 Discount: 30% off
5 hours left at this price!
Buy now
30-Day Money-Back Guarantee
This course includes
  • 3 hours on-demand video
  • 3 articles
  • 15 downloadable resources
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • This course will provide you an in depth knowledge of apache Spark and how to work with spark using Azure Databricks.
  • You will learn to Provision your own Databricks workspace using Azure cloud.
  • You will be able to create application on Azure Databricks after completing the course
  • You will be able to process continual streams of data with Spark streaming using azure event Hub
  • Transform structured data using SparkSQL and Dataframes
  • Build a binary classification application using the Mlib pipelines API.
Course content
Expand all 34 lectures 03:00:18
+ – Overview
2 lectures 05:04

An overview of our course

Course Overview
Preview 02:30

Something about us … the BIG DATA TRUNK

Big Data trunk Introduction !!
02:34
+ – Apache Spark Introduction
9 lectures 01:05:08

How we use to work with big Data before the introduction of Spark

Big Data before Spark
Preview 11:42

How the Big data world drastically changed after the introduction of Spark

Big Data After Spark
07:30

An Introduction to APACHE SPARK

Introduction to Apache Spark
04:11

Introduction to Spark RDD

Spark - RDD
06:01
Lab - Ways for creating RDD
06:32
Quiz: Spark Introduction
4 questions

Brief Discussion about Spark Architecture

Spark Application Architecture
04:52

Types of Operations done on Spark RDD. We will mainly discuss about Transformation and actions. We will also discuss few sample codes to illustrate in a better way.

Spark Operations - Transformation & Action
07:18
Lab - Transformation & Action
07:42
DAG (Direct Acyclic Graph)
09:20
Quiz-2
1 question
+ – Databricks with Microsoft Azure
4 lectures 19:49

An introduction to Azure Databricks

Spark Databricks Analytics in Azure
06:36

Here in video we will briefly discuss why we need Azure Databricks even if we have Apache Spark

Why Databricks ?
05:24

We will provision Azure Databricks cluster and also we will learn to create a cluster in Azure Databricks

Provisioning Databricks workspace in Azure portal
03:15
Using Databricks community Edition
04:34
Quiz-3
2 questions
+ – Understanding Cluster and Notebooks in Databricks
1 lecture 03:18

We will learn how we can use notebook in databricks and how it can be associated to any Cluster in Databricks

Use notebook in Databricks
03:18
+ – Working with Spark in Databricks
6 lectures 52:28

We will use Jupiter notebook to run queries on set of external data. Also we will learn to use SQL & Panda queries in notebook.

After that we will learn to plot different types of charts

Spark SQL
08:01
Quiz-4
2 questions

We will Develop a word count program. Also we will run this program on set of external data by removing special characters from the dataset.

Developing Spark application
10:28
Removing Stopwords
02:00

In this video we will cover Databricks runtime Machine learning. We will use Binay Composition and decision tree model to train our Model

Developing Spark ML application
14:06

We will work on Streaming data from the twitter. We will use Azure eventHub for reading the Twits from Twitter.

Developing Spark Streaming applications (real time Twitter Data)
07:47

How the Spark can be optimized to perform more efficiently and effectively.

Optimizing Spark Performance
10:06
Quiz-5
1 question
+ – Spark Interview
10 lectures 33:29
Spark Vs Hadoop / mapReduce
Preview 03:50
Spark Core
01:59
Something more about RDD
05:43
Type Safety in Dataset
03:16
Justify Lazy Evaluation in Spark
02:15
Explain Fault Tolerance in Spark
05:44
Coalesce and Repartition
02:57
Spark Executor Memory
01:06
Garbage Collection Optimization in Spark
06:33
Learn More on Spark Interview
00:06
+ – Bonus Section
2 lectures 01:01
Thanks You looking way forward !!
00:40
Bonus Lecture
00:21
Requirements
  • Some Prior scripting knowledge, anyways we will be explainig all the codes line by line whichever we will be using in our labs
  • Free or paid subscription for Microsoft Azure portal.
Description

In this course you will learn the basics of creating Spark jobs, loading data, and working with data. You’ll also get an introduction to running machine learning algorithms and working with streaming data. Databricks lets you start writing Spark queries instantly so you can focus on your data problems.

Azure Databricks accelerate big data analytics and artificial intelligence (AI) solutions, a fast, easy and collaborative Apache Spark–based analytics service.

Why Azure Databricks?

Productive : Launch your new Apache Spark environment in minutes.

Scalable : Globally scale your analytics and machine learning projects.

Trusted : Help protect your data and business with Azure AD integration, role-based controls and enterprise-grade SLAs.

Flexible : Build machine learning and AI solutions with your choice of language and deep learning frameworks.


We believe that when you learn something, you should be able to apply it somewhere. So, in this course, we are also providing you with some of the important spark interview questions , which will help you to crack the interview with flying colors.

Who this course is for:
  • Anyone who wants to learn Spark using Azure Databricks