What you'll learn

Entire curriculum of CCA Spark and Hadoop Developer
Apache Sqoop
HDFS Commands
Scala Fundamentals
Core Spark - Transformations and Actions
Spark SQL and Data Frames
Streaming analytics using Kafka, Flume and Spark Streaming

Requirements

Basic programming skills
Cloudera Quickstart VM or valid account for IT Versity Big Data labs or any Hadoop clusters where Hadoop, Hive and Spark are well integrated.
Minimum memory required based on the environment you are using with 64 bit operating system

Description

CCA 175 Spark and Hadoop Developer is one of the well recognized Big Data certification. This scenario based certification exam demands basic programming using Python or Scala along with Spark and other Big Data technologies.

This comprehensive course covers all aspects of the certification using Scala as programming language.

Scala Fundamentals
Core Spark - Transformations and Actions
Spark SQL and Data Frames
File formats
Flume, Kafka and Spark Streaming
Apache Sqoop

Exercises will be provided to prepare before attending the certification. Intention of the course is to boost the confidence to attend the certification.

All the demos are given on our state of the art Big Data cluster. You can avail one week complementary lab access by filling the form which is provided as part of the welcome message.

Who this course is for:

Any IT aspirant/professional willing to learn Big Data and give CCA 175 certification

Featured review

Pavol Vadkerti

24 courses

12 reviews

a year ago

Really one of the "from zero to hero" courses. I was typing all the commands like in the video, making notes and had an ITversity lab account and still it took me 4 Months to finish the course (did it in my free time). The instructor was very skilled, easy to understand. I'm looking forward to the next course. Big thumb up!

11 sections • 161 lectures • 27h 52m total length

Preview08:01

Preview10:51
Setup Scala on Windows
07:23
Basic Programming Constructs
18:53
Functions
18:35
Object Oriented Concepts - Classes
17:42
Object Oriented Concepts - Objects
13:02
Object Oriented Concepts - Case Classes
11:14
Collections - Seq, Set and Map
08:56
Basic Map Reduce Operations
14:08
Setting up Data Sets for Basic I/O Operations
04:23
Basic I/O Operations and using Scala Collections APIs
16:23
Tuples
04:56
Development Cycle - Developing Source code
07:24
Development Cycle - Compile source code to jar using SBT
09:32
Development Cycle - Setup SBT on Windows
02:48
Development Cycle - Compile changes and run jar with arguments
04:21
Development Cycle - Setup IntelliJ with Scala
12:07
Development Cycle - Develop Scala application using SBT in IntelliJ
10:50

Introduction
05:15
Introduction to Spark
02:22
Setup Spark on Windows
23:15
Quick overview about Spark documentation
04:49
Initializing Spark job using spark-shell
18:39
Create Resilient Distributed Data Sets (RDD)
13:40
Previewing data from RDD
17:57
Reading different file formats - Brief overview using JSON
09:34
Transformations Overview
04:02
Manipulating Strings as part of transformations using Scala
13:44
Row level transformations using map
18:09
Row level transformations using flatMap
09:19
Filtering the data
18:03
Joining data sets - inner join
10:34
Joining data sets - outer join
17:29
Aggregations - Getting Started
04:07
Aggregations - using actions (reduce and countByKey)
15:14
Aggregations - understanding combiner
06:50
Aggregations using groupByKey - least preferred API for aggregations
21:13
Aggregations using reduceByKey
07:36
Aggregations using aggregateByKey
18:21
Sorting data using sortByKey
19:35
Global Ranking - using sortByKey with take and takeOrdered
12:47
By Key Ranking - Converting (K, V) pairs into (K, Iterable[V]) using groupByKey
06:21
Get topNPrices using Scala Collections API
10:49
Get topNPricedProducts using Scala Collections API
11:29
Get top n products by category using groupByKey, flatMap and Scala function
06:02
Set Operations - union, intersect, distinct as well as minus
19:39
Save data in Text Input Format
15:13
Save data in Text Input Format using Compression
11:28
Saving data in standard file formats - Overview
10:23
Revision of Problem Statement and Design the solution
04:12
Solution - Get Daily Revenue per Product - Launching Spark Shell
10:08
Solution - Get Daily Revenue per Product - Read and join orders and order_items
17:46
Solution - Get Daily Revenue per Product - Compute daily revenue per product id
13:41
Solution - Get Daily Revenue per Product - Read products data and create RDD
15:22
Solution - Get Daily Revenue per Product - Sort and save to HDFS
26:17
Solution - Add spark dependencies to sbt
08:01
Solution - Develop as Scala based application
25:34
Solution - Run locally using spark-submit
09:03
Solution - Ship and run it on big data cluster
13:21

Different interfaces to run Hive queries
09:25
Create Hive tables and load data in text file format
25:00
Create Hive tables and load data in ORC file format
10:18
Using spark-shell to run Hive queries or commands
03:51
Functions - Getting Started
05:11
Functions - Manipulating Strings
22:23
Functions - Manipulating Dates
13:44
Functions - Aggregations
05:49
Functions - CASE
14:10
Row level transformations
08:30
Joins
18:10
Aggregations
11:41
Sorting
07:27
Set Operations
05:39
Analytics Functions - Aggregations
15:53
Analytics Functions - Ranking
08:39
Windowing Functions
07:48
Create Data Frame and Register as Temp table
15:40
Writing Spark SQL Applications - process data
08:38
Writing Spark SQL Applications - Save data into Hive tables
07:20
Data Frame Operations
12:54

Introduction to Pre-defined Functions
05:51
Creating Spark Session Object in Notebook
01:55
Create Dummy Data Frames for Practice
08:06
Categories of Functions
02:16
Using Special Functions - col
13:50
Using Special Functions - lit
04:44
String Manipulation Functions - Case Conversion and Length
06:44
String Manipulation - Extracting data from fixed lengith fields using substring
13:16
String Manipulation - Extracting data from delimited fields using split
09:04
String Manipulation - Concatenating Strings
03:37
String Manipulation - Padding Strings
11:10
String Manipulation - Trimming unwanted characters
05:23
Date and Time Functions - Overview
04:14
Date and Time Functions - Date Arithmetic
09:53
Date and Time Functions - Using trunc and date_trunc for to date reports
07:34
Date and Time Functions - Using date_format and other functions
15:33
Date and Time Functions - dealing with unix timestamp
08:13
Pre-defined Functions - Conclusion
04:23

Introduction to Basic Transformations using Data Frame APIs
02:51
Starting Spark Context
03:13
Overview of Filtering
05:24
Filtering - Reading Data and Understanding Schema
02:30
Filtering Data - Task 1 - Equal Operator
08:19
Filtering Data - Task 2 - Comparison Operators
03:41
Filtering Data - Task 3 - Boolean AND
05:22
Filtering Data - Task 4 - IN Operator
05:43
Filtering Data - Task 5 - Between and Like
09:09
Filtering Data - Task 6 - Using functions in Filter
09:48
Overview of Aggregations
08:41
Overview of Sorting
02:52
Solution - Get Delayed Counts - Part 1
06:47
Solution - Get Delayed Counts - Part 2
05:22
Solution - Getting Delayed Counts By Date
16:28

Prepare and Validate Data Sets
04:43
Starting Spark Session or Spark Context
03:28
Analyze Data Sets for Joins
06:11
Eliminate Duplicate records from Data Frame
04:11
Recap of Basic Transformations
04:27
Joining Data Sets - Problem Statements
02:11
Overview of Joins
01:43
Inner Join - Get number of flights departed from US airports
09:17
Inner Join - Get number of flights departed from US States
05:08
Outer Join - Get Aiports - Never Used
07:44

Durga Viswanatha Raju Gadiraju

Technology Adviser and Evangelist

4.1 Instructor Rating
9,306 Reviews
159,846 Students
19 Courses

13+ years of experience in executing complex projects using vast array of technologies including Big Data and Cloud.

I found itversity, llc - a US based startup to provide quality training for IT professionals and staffing as well as consulting solutions for enterprise clients. I have trained thousands of IT professionals in vast array of technologies including Big Data and Cloud.

Building IT career for people and provide quality services to the clients will be paramount to our organization.

As an entry strategy itversity will be providing quality training in the areas of ABCD

* Application Development
* Big Data and Business Intelligence
* Cloud
* Datawarehousing, Databases

Itversity Support

Support Account for ITVersity Courses.

4.1 Instructor Rating
9,306 Reviews
158,723 Students
19 Courses

We have built a team to support going forward. If you send messages to this account for our courses, they will be sent to our Helpdesk from where we will be rewriting to our team.

Kamlesh Patidar

KP

4.1 Instructor Rating
2,671 Reviews
20,078 Students
4 Courses

This course includes:

Training 5 or more people?

CCA 175 - Spark and Hadoop Developer Certification - Scala

What you'll learn

Requirements

Description

Who this course is for:

Featured review

Course content

Instructors

This course includes:

Training 5 or more people?

CCA 175 - Spark and Hadoop Developer Certification - Scala

What you'll learn

Requirements

Description

Who this course is for:

Featured review

Course content

Introduction1 lecture • 8min

Scala Fundamentals18 lectures • 3hr 13min

Getting Started9 lectures • 1hr 21min

Transform, Stage and Store - Spark41 lectures • 8hr 47min

Data Analysis - Spark SQL or HiveQL - 1.6.x21 lectures • 3hr 58min

Setup Hadoop and Spark Environment for Practice8 lectures • 43min

Spark 2 - Data Processing - Overview7 lectures • 1hr 18min

Spark 2 - Processing Column Data using Pre-defined Functions18 lectures • 2hr 16min

Spark 2 - Basic Transformations using Data Frames15 lectures • 1hr 36min

Spark 2 - Joining Data Sets10 lectures • 49min

Instructors