What you'll learn

Entire curriculum of CCA Spark and Hadoop Developer
HDFS Commands
Scala Fundamentals
Core Spark - Transformations and Actions
Spark SQL and Data Frames

Requirements

Basic programming skills
Cloudera Quickstart VM or valid account for IT Versity Big Data labs or any Hadoop clusters where Hadoop, Hive and Spark are well integrated.
Minimum memory required based on the environment you are using with 64 bit operating system

Description

CCA 175 Spark and Hadoop Developer is one of the well recognized Big Data certification. This scenario based certification exam demands basic programming using Python or Scala along with Spark and other Big Data technologies.

This comprehensive course covers all aspects of the certification using Scala as programming language.

Scala Fundamentals
Core Spark - Transformations and Actions
Spark SQL and Data Frames
File formats
Flume, Kafka and Spark Streaming
Apache Sqoop

Exercises will be provided to prepare before attending the certification. Intention of the course is to boost the confidence to attend the certification.

All the demos are given on our state of the art Big Data cluster. You can avail one week complementary lab access by filling the form which is provided as part of the welcome message.

Who this course is for:

Any IT aspirant/professional willing to learn Big Data and give CCA 175 certification

Featured review

Pavol Vadkerti

25 courses

12 reviews

a year ago

Really one of the "from zero to hero" courses. I was typing all the commands like in the video, making notes and had an ITversity lab account and still it took me 4 Months to finish the course (did it in my free time). The instructor was very skilled, easy to understand. I'm looking forward to the next course. Big thumb up!

15 sections • 196 lectures • 29h 3m total length

Preview08:01

Preview10:51
Setup Scala on Windows
07:23
Basic Programming Constructs
18:53
Functions
18:35
Object Oriented Concepts - Classes
17:42
Object Oriented Concepts - Objects
13:02
Object Oriented Concepts - Case Classes
11:14
Collections - Seq, Set and Map
08:56
Basic Map Reduce Operations
14:08
Setting up Data Sets for Basic I/O Operations
04:23
Basic I/O Operations and using Scala Collections APIs
16:23
Tuples
04:56
Development Cycle - Developing Source code
07:24
Development Cycle - Compile source code to jar using SBT
09:32
Development Cycle - Setup SBT on Windows
02:48
Development Cycle - Compile changes and run jar with arguments
04:21
Development Cycle - Setup IntelliJ with Scala
12:07
Development Cycle - Develop Scala application using SBT in IntelliJ
10:50

Introduction
05:15
Introduction to Spark
02:22
Setup Spark on Windows
23:15
Quick overview about Spark documentation
04:49
Initializing Spark job using spark-shell
18:39
Create Resilient Distributed Data Sets (RDD)
13:40
Previewing data from RDD
17:57
Reading different file formats - Brief overview using JSON
09:34
Transformations Overview
04:02
Manipulating Strings as part of transformations using Scala
13:44
Row level transformations using map
18:09
Row level transformations using flatMap
09:19
Filtering the data
18:03
Joining data sets - inner join
10:34
Joining data sets - outer join
17:29
Aggregations - Getting Started
04:07
Aggregations - using actions (reduce and countByKey)
15:14
Aggregations - understanding combiner
06:50
Aggregations using groupByKey - least preferred API for aggregations
21:13
Aggregations using reduceByKey
07:36
Aggregations using aggregateByKey
18:21
Sorting data using sortByKey
19:35
Global Ranking - using sortByKey with take and takeOrdered
12:47
By Key Ranking - Converting (K, V) pairs into (K, Iterable[V]) using groupByKey
06:21
Get topNPrices using Scala Collections API
10:49
Get topNPricedProducts using Scala Collections API
11:29
Get top n products by category using groupByKey, flatMap and Scala function
06:02
Set Operations - union, intersect, distinct as well as minus
19:39
Save data in Text Input Format
15:13
Save data in Text Input Format using Compression
11:28
Saving data in standard file formats - Overview
10:23
Revision of Problem Statement and Design the solution
04:12
Solution - Get Daily Revenue per Product - Launching Spark Shell
10:08
Solution - Get Daily Revenue per Product - Read and join orders and order_items
17:46
Solution - Get Daily Revenue per Product - Compute daily revenue per product id
13:41
Solution - Get Daily Revenue per Product - Read products data and create RDD
15:22
Solution - Get Daily Revenue per Product - Sort and save to HDFS
26:17
Solution - Add spark dependencies to sbt
08:01
Solution - Develop as Scala based application
25:34
Solution - Run locally using spark-submit
09:03
Solution - Ship and run it on big data cluster
13:21

Introduction to Pre-defined Functions
05:51
Creating Spark Session Object in Notebook
01:55
Create Dummy Data Frames for Practice
08:06
Categories of Functions
02:16
Using Special Functions - col
13:50
Using Special Functions - lit
04:44
String Manipulation Functions - Case Conversion and Length
06:44
String Manipulation - Extracting data from fixed lengith fields using substring
13:16
String Manipulation - Extracting data from delimited fields using split
09:04
String Manipulation - Concatenating Strings
03:37
String Manipulation - Padding Strings
11:10
String Manipulation - Trimming unwanted characters
05:23
Date and Time Functions - Overview
04:14
Date and Time Functions - Date Arithmetic
09:53
Date and Time Functions - Using trunc and date_trunc for to date reports
07:34
Date and Time Functions - Using date_format and other functions
15:33
Date and Time Functions - dealing with unix timestamp
08:13
Pre-defined Functions - Conclusion
04:23

Introduction to Basic Transformations using Data Frame APIs
02:51
Starting Spark Context
03:13
Overview of Filtering
05:24
Filtering - Reading Data and Understanding Schema
02:30
Filtering Data - Task 1 - Equal Operator
08:19
Filtering Data - Task 2 - Comparison Operators
03:41
Filtering Data - Task 3 - Boolean AND
05:22
Filtering Data - Task 4 - IN Operator
05:43
Filtering Data - Task 5 - Between and Like
09:09
Filtering Data - Task 6 - Using functions in Filter
09:48
Overview of Aggregations
08:41
Overview of Sorting
02:52
Solution - Get Delayed Counts - Part 1
06:47
Solution - Get Delayed Counts - Part 2
05:22
Solution - Getting Delayed Counts By Date
16:28

Prepare and Validate Data Sets
04:43
Starting Spark Session or Spark Context
03:28
Analyze Data Sets for Joins
06:11
Eliminate Duplicate records from Data Frame
04:11
Recap of Basic Transformations
04:27
Joining Data Sets - Problem Statements
02:11
Overview of Joins
01:43
Inner Join - Get number of flights departed from US airports
09:17
Inner Join - Get number of flights departed from US States
05:08
Outer Join - Get Aiports - Never Used
07:44

Getting Started - Overview
02:01
Overview of Spark Documentation
02:29
Launching and using Spark SQL CLI
04:08
Overview of Spark SQL Properties
08:51
Running OS Commands using Spark SQL
03:19
Understanding Warehouse Directory
04:12
Managing Spark Metastore Databases
10:01
Managing Spark Metastore Tables
03:21
Retrieve Metadata of Tables
02:19
Role of Spark Metastore or Hive Metastore
05:01
Exercise - Getting Started with Spark SQL
08:57

Durga Viswanatha Raju Gadiraju

Technology Adviser and Evangelist

4.2 Instructor Rating
9,026 Reviews
166,907 Students
18 Courses

13+ years of experience in executing complex projects using vast array of technologies including Big Data and Cloud.

ITVersity, Inc. - a US based organization to provide quality training for IT professionals and we have the track record of training hundreds of thousands of professionals globally.

Building IT career for people with required tools such as high quality material, labs, live support etc to upskill and cross skill is paramount for our organization.

At this time our training offerings are focused on following areas:

* Application Development using Python and SQL

* Big Data and Business Intelligence

* Cloud

* Datawarehousing, Databases

Itversity Support

Support Account for ITVersity Courses.

4.2 Instructor Rating
8,384 Reviews
151,137 Students
15 Courses

We have built a team to support going forward. If you send messages to this account for our courses, they will be sent to our Helpdesk from where we will be rewriting to our team.

Hindu Varma Datla

Software Engineer at ITVersity

4.2 Instructor Rating
3,855 Reviews
45,867 Students
4 Courses

3+ years of IT Experience in the areas of Python using Django as well as Flask, Spark, Linux, SQL using any RDBMS, Java Script, Node JS, Mongo DB etc.

I will be primarily providing support for Python, SQL and other related courses as co-instructor to ITVersity courses.

ITVersity, Inc. - a US based organisation to provide quality training for IT professionals and we have the track record of training hundreds of thousands of professionals globally.

Building IT career for people with required tools such as high quality material, labs, live support etc to up skill and cross skill is paramount for our organisation.

At this time our training offerings are focused on following areas:

* Application Development using Python and SQL

* Big Data and Business Intelligence

* Cloud

* Data Warehousing, Databases

Teja Rayala

Software Engineer at ITVersity Inc.

TR

4.3 Instructor Rating
3,675 Reviews
23,073 Students
3 Courses

Experienced Data Engineer with a demonstrated history of working in the consumer goods industry. Skilled in Apache Airflow, Apache Kafka, Hive, Apache Spark, and Amazon Web Services (AWS). Strong information technology professional with a Master's degree focused in Analytics from University of Cincinnati.

ITVersity, Inc. is a US-based organisation providing quality training for IT professionals and we have a track record of training hundreds of thousands of professionals globally.

Helping build IT careers of people with high-quality content, Labs, live support etc. to upskill and cross-skill is paramount for our organisation.

I will be overseeing the support for ITVersity courses related to Data Engineering and DevOps Engineering

At this time our training offerings are focused on the following areas:

* Application Development using Python and SQL

* Big Data and Business Intelligence

* Cloud

* Data Warehousing, Databases

This course includes:

CCA 175 - Spark and Hadoop Developer Certification - Scala

What you'll learn

Requirements

Description

Who this course is for:

Featured review

Course content

Instructors

This course includes:

CCA 175 - Spark and Hadoop Developer Certification - Scala

What you'll learn

Requirements

Description

Who this course is for:

Featured review

Course content

Introduction1 lecture • 8min

Scala Fundamentals18 lectures • 3hr 13min

Getting Started9 lectures • 1hr 21min

Apache Spark 1. 6 - Transform, Stage and Store - Spark41 lectures • 8hr 47min

Setup Hadoop and Spark Environment for Practice8 lectures • 43min

Spark 2 - Data Processing - Overview7 lectures • 1hr 18min

Spark 2 - Processing Column Data using Pre-defined Functions18 lectures • 2hr 16min

Spark 2 - Basic Transformations using Data Frames15 lectures • 1hr 36min

Spark 2 - Joining Data Sets10 lectures • 49min

Apache Spark using SQL - Getting Started11 lectures • 55min

Instructors