Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

Working with Apache Spark (Aug 2023)

Name: Working with Apache Spark (Aug 2023)
Rating: 4.4 (6 reviews)

A Complete Primer for Apache Spark

Created byProton Expert Systems & Solutions

Last updated 9/2023

English

What you'll learn

Apache Spark and its features
Installing and Configuring Spark Programming Environment
Spark Programming using Scala
Creating and Working with Spark Context, Spark RDD, DataFrames, DataSets
Transformations and Actions using DataFrames
Spark SQL, Spark Streaming with Kafka, GraphX, Spark Mllib, PySpark and Sparklyr
Scheduling Spark Jobs

Course content

9 sections • 28 lectures • 2h 58m total length

Introduction5:37
This Course includes all the features of Apache Spark both as standalone and on Cloudera Stack.

Lesson 01: Introduction to Spark18:14
In this Lesson, Features of Spark 3.0 is discussed along with the difference between Hadoop and Spark. Also Hadoop Ecosystem is discussed.
Practice 1-1: Installing and Configuring Standalone Spark Computing Environment10:20
In this practice, you will download and install the prerequisites for Spark. Further, download and install Scala IDE.
Practice 1-2: Create & Run a Project on Standalone Spark Programming Environment2:25
In this practice, you will learn to develop a simple First Project and run it on your local machine.
Quiz on Introduction to Spark

Lesson 02: Spark Runtime: Context, Executor and Stage11:17
Here, you will get know about Spark Runtime Environment. How to Create Spark Context and how to use it in spark program is discussed in detail.
Practice 2-1: Starting Spark on Cloudera Hadoop Ecosystem4:22
In this practice, you will download the Cloudera, start and connect to the Cloudera using Docker.
Practice 2-2: Exploring the Spark Context in the Cloudera Ecosystem2:43
In this practice, you will create Spark Context program, execute and perform different functions using Spark on Cloudera.
Quiz on Spark Runtime: Contact, Executor and Stage

Lesson 03: Working with Spark RDD10:52
Creating Spark RDD, using Transformation operations with actions is illustrated. Features of Paired RDDs, Persistence Levels and Lazy Evaluation is covered.
Practice 3-1: Working with RDD Transformations and Actions6:26
In this practice, you will learn about Different RDD transformations and actions.
Practice 3-2: Working with Paired RDDs2:09
In this practice, you will learn about different operations on paired RDD.
Practice 3-3: Using Cache and Persist Methods in Spark2:05
In this practice, you will learn about Cache and Persist methods.
Quiz on Working with Spark RDD

Lesson 04: Working with Spark SQL10:39
In this lesson, you will Learn about Spark SQL Libraries, Features of Spark SQL, how to create Data Frames and perform various operations on it.
Practice 4-1: DataFrames in Spark SQL4:30
In this practice, you will learn about DataFrame and Dataset in Spark SQL.
Practice 4-2: DataSets in Spark SQL1:20
In this practice, you will learn about Dataset in Spark SQL.
Practice 4-3: Using Window Function in Spark SQL5:54
In this practice, you will learn about Spark windowing function.
Practice 4-4: Creating DataFrame and Dataset in Standalone Spark Environment1:48
In this practice, you will learn to create DataFrame and dataset in the Standalone Spark Programming Environment.
Quiz on Working with Spark SQL

Lesson 05: Working with Spark Streaming13:31
Elements of Spark Streaming, DStream operations, Transformations in Spark Streaming, Streaming using Apache Kafka are discussed in detail.
Practice 5-1: Installing and Running Apache Kafka in a Standalone Environment7:12
In this practice, you will install and configure Apache Zookeeper along with Apache Kafka. Further, you will create a sample Topic in Kafka and test by sending and receiving messages using producer and consumer.
Practice 5-2: Spark Streaming with Apache Kafka8:20
In this practice, you will learn spark streaming with Apache Kafka by considering it as input data source.
Quiz on Working with Spark Streaming

Lesson 06: Using Sparklyr6:11
Using Sparklyr package in R programming is illustrated with example. Using Sparklyr processing DataFrames in R is discussed.
Practice 6-1: Configuring R Programming Environment3:27
In this practice, you will install and configure R, RStudio and Rtools.
Practice 6-2: Working with Spark DataFrames in R3:47
In this practice, you will create Spark DataFrame and perform various operations on the DataFrames from R program.
Quiz on Using Sparklyr

Lesson 07: Advanced Features with Spark10:52
In this lesson, you will learn programming using PySpark. You will also learn about GraphX and MLlib.
Practice 7-1: Performing various Spark Operations using Pyspark10:53
In this practice, you will create SparkContext in PySpark and use various PySpark RDD functions. Also use the SQLContext, HIVEContext and PySpark DataFrames.
Practice 7-2: Working with GraphX4:29
In this practice, you will use various spark GraphX functions.
Practice 7-3: Implementing Linear Regression using MLlib3:35
In this practice, you will implement Linear Regression Machine Learning Algorithm using Spark.
Quiz on Advanced Features with Spark

Requirements

Working Knowledge on Cloudera Hadoop Stack
Basic Programming Knowledge
Basic Linux Commands

Description

In this Course, you will Learn in detail about Apache Spark and its Features. This is course deep dives into Features of Apache Spark, RDDs, Transformation, Actions, Lazy Execution, Data Frames, DataSets, Spark SQL, Spark Streaming, PySpark, Sparklyr and Spark Jobs.

You will explore creating Spark RDD and performing various transformation operations on RDDs along with actions. This Course also illustrates the difference between RDD, DataFrame and DataSet with examples. You will also explore features of Spark SQL and execute database queries using various contexts.

In this course, you will also explore Spark Streaming along with Kafka. The Spark Streaming examples includes producing and consuming messages on a Kafka Topic. Spark program is basically coded using Scala in this course, but PySpark is also discussed, programming examples using PySpark is also included.

Usage of Sparklyr package in R Programming is included in the Course. Finally, the course includes how to schedule and execute Spark Jobs.

The course teaches you everything you need to know about Apache Spark.

This course gives details about Working with Apache Spark with an emphasis on its activity lessons and hands on experience.

What are you waiting for?

Every day is a missed opportunity.

Enroll No!

Hurry up!

Who this course is for:

Data Scientists / Data Engineers
Big Data Developers
Big Data Engineers
Big Data Architects
Any technical personnel who are interested in learning and Exploring the Features of Apache Spark

Working with Apache Spark (Aug 2023)

What you'll learn

Explore related topics

Course content

Introduction1 lecture • 6min

Introduction to Spark3 lectures • 31min

Spark Runtime: Context, Executor and Stage3 lectures • 18min

Working with Spark RDD4 lectures • 22min

Working with Spark SQL5 lectures • 24min

Working with Spark Streaming3 lectures • 29min

Using Sparklyr3 lectures • 13min

Advanced Features with Spark4 lectures • 30min

Executing and Scheduling the Spark job2 lectures • 5min

Requirements

Description

Who this course is for: