A Big Data Hadoop and Spark project for absolute beginners
What you'll learn
- Big Data , Hadoop and Spark from scratch using Python and Scala. You will also learn how to use free cloud tools to get started with Hadoop and Spark programming in minutes. Additionally you will find two bonus projects on AWS data lake solution and Machine Learning Classification model
Requirements
- Students should have some programming background and some knowledge of SQL queries.
Description
A bank is launching a new credit card and wants to identify prospects it can target in its marketing campaign.
It has received prospect data from various internal and 3rd party sources. The data has various issues such as missing or unknown values in certain fields.The data needs to be cleansed before any kind of analysis can be done.
Since the data is in huge volume with billions of records, the bank has asked you to use Big Data Hadoop and Spark technology to cleanse, transform and analyze this data.
What you will learn :
Big Data, Hadoop concepts
How to create a free Hadoop and Spark cluster using Google Dataproc
Hadoop hands-on - HDFS, Hive
Why there was a need for Spark
Python basics
PySpark RDD - hands-on
PySpark SQL, DataFrame - hands-on
Project work using PySpark and Hive
Scala basics
Spark Scala DataFrame
Project working using Spark Scala
Google Colab environment
Bonus project - Applying spark transformation on data stored in AWS S3 using Glue and viewing data using Athena
Prerequisites :
Some basic programming skills
Some knowledge of SQL queries
Who this course is for:
- Beginners who want to learn Big Data or experienced people who want to transition to a Big Data role
Course content
- Preview03:32
Instructor
![FutureX Skill](data:image/svg+xml,%3Csvg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64"%3E%3C/svg%3E){kind=avatar}
- 4.1 Instructor Rating
- 632 Reviews
- 19,304 Students
- 6 Courses
We are a group of Solution Architects and Developers with expertise in Java, Python, Scala , Big Data , Machine Learning and Cloud.
We have years of experience in building Data and Analytics solutions for global clients.
Our primary goal is to simplify learning for our students.
We take a very practical use case based approach in all our courses.