Mastering Apache SQOOP with Hadoop,Hive, MySQL (Mac & Win)
3.9 (31 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
328 students enrolled

Mastering Apache SQOOP with Hadoop,Hive, MySQL (Mac & Win)

The Complete Course on Apache SQOOP. Great for CCA175 Spark & Hortonworks Big Data Hadoop Developer Certifications.
3.9 (31 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
328 students enrolled
Created by DataShark Academy
Last updated 12/2018
English
English [Auto-generated]
Current price: $139.99 Original price: $199.99 Discount: 30% off
5 hours left at this price!
30-Day Money-Back Guarantee
This course includes
  • 3.5 hours on-demand video
  • 5 articles
  • 4 downloadable resources
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Get Ready for CCA Spark and Hadoop Developer Exam (CCA175)
  • Get Ready for Hortonworks Data Platform (HDP) Certified Developer Exam (HDPCD)
  • Advance your career by applying for high paying Big Data jobs
  • Install & configure Hortonworks Data Platform (HDP) Sandbox on Windows Machine
  • Crack Big Data Developer Interviews
  • Develop sound understanding about Data Ingestion process from Relational System (MySQL) to Hadoop ecosystem & vice versa
Requirements
  • Basic knowledge of computers and SQL queries will help. Detailed explanations are provided wherever felt needed in the course.
Description

WHY APACHE SQOOP

Apache SQOOP is designed to import data from relational databases such as Oracle, MySQL, etc to Hadoop systems. Hadoop is ideal for batch processing of huge amounts of data. It is industry standard nowadays. In real world scenarios, using SQOOP you can transfer the data from relational tables into Hadoop and then leverage the parallel processing capabilities of Hadoop to process huge amounts of data and generate meaningful data insights. The results of Hadoop processing can again be stored back to relational tables using SQOOP export functionality. 


Big data analytics start with data ingestion and thats where apache sqoop comes in picture. It is the first step in getting the data ready.


ABOUT THIS COURSE

In this course, you will learn step by step everything that you need to know about Apache Sqoop and how to integrate it within Hadoop ecosystem. With every concept explained with real world like examples, you will learn how to create Data Pipelines to move in/out the data from Hadoop. In this course, you will learn following major concepts in great details:


APACHE SQOOP - IMPORT TOPICS   << MySQL to Hadoop/Hive >>

  1. default hadoop storage

  2. specific target on hadoop storage

  3. controlling parallelism

  4. overwriting existing data

  5. append data

  6. load specific columns from MySQL table

  7. control data splitting logic

  8. default to single mapper when needed

  9. Sqoop Option files

  10. debugging Sqoop Operations

  11. Importing data in various file formats - TEXT, SEQUENCE, AVRO, PARQUET & ORC

  12. data compression while importing

  13. custom query execution

  14. handling null strings and non string values

  15. setting delimiters for imported data files

  16. setting escaped characters

  17. incremental loading of data

  18. write directly to hive table

  19. using HCATALOG parameters

  20. importing all tables from MySQL database

  21. importing entire MySQL database into Hive database


APACHE SQOOP - EXPORT TOPICS  << Hadoop/Hive to MySQL >>

  1. Move data from Hadoop to MySQL table

  2. Move specific columns from Hadoop to MySQL table

  3. Avoid partial export issues

  4. Update Operation while exporting


APACHE SQOOP - JOBS TOPICS  << Automation >>

  1. create sqoop job

  2. list existing sqoop jobs

  3. check metadata about sqoop jobs

  4. execute sqoop job

  5. delete sqoop job

  6. enable password storage for easy execution in production


WHAT YOU WILL ACHIEVE AFTER COMPLETING THIS COURSE

After completing this course, you will cover one of the topic that is heavily asked in below certifications. You will need to take other lessons as well to fully prepare for the test. We will be launching other courses soon.

1. CCA Spark and Hadoop Developer Exam (CCA175)

2. Hortonworks Data Platform (HDP) Certified Developer Exam (HDPCD)


WHO ARE YOUR INSTRUCTORS

This course is taught by professionals with extensive experience in handling big data applications for Fortune 100 companies of the world. They have managed to create data pipelines for extracting, transforming & processing over 100's of Terabytes of data in a day for their clients providing data analytics for user services. After successful launch of their course - Complete ElasticSearch with LogStash, Hive, Pig, MR & Kibana, same team has brought to you a complete course on learning Apache Sqoop with Hadoop, Hive, MySQL.


You will also get step by step instructions for installing all required tools and components on your machine in order to run  all examples provided in this course. Each video will explain entire process in detail and easy to understand manner.

You will get access to working code for you to play with it and expand on it. All code examples are working and will be demonstrated in video lessons.

Windows users will need to install virtual machine on their device to setup single node hadoop cluster while MacBook or Linux users can directly install hadoop and sqoop components on their machines. The step by step process is illustrated within course.

Who this course is for:
  • This will be an excellent course for anyone who wants to learn Big Data technologies.
  • Anyone looking to pass CCA 175 Spark Certification exam in future
  • Anyone looking to pass Hortonworks Data Platform (HDP) Certified Developer Exam (HDPCD)
Course content
Expand all 47 lectures 03:33:46
+ Environment Setup
8 lectures 10:32
Install Hortonworks Data Platform Sandbox - ( FOR WINDOWS PC USERS ONLY )
00:04
Install Hadoop & SQOOP on Machine - ( FOR MAC/LINUX USERS)
00:03
Connect to HDP Sandbox Shell
00:03
Get to know SQOOP CLI
02:03

Follow the instructions to load data for exercise.

Load Data into MySQL Database
01:20
Data Setup for Exercises
01:56
Let's Understand Your Data
01:56
+ Apache SQOOP - IMPORT
26 lectures 02:19:40
Import a Simple MySQL Table into Hadoop HDFS
06:24
Import a MySQL Table with Custom Name into Hadoop
04:14
Controlling Paralellism in SQOOP Import Flow
04:47
Overwrite Existing Data on Hadoop while Importing
07:31
Append to Existing Data on Hadoop while Importing
04:07
Only load specific columns from MySQL table into Hadoop
03:34
Import MySQL tables with No Primary keys in them - 1st Approach
05:54
Import MySQL tables with No Primary keys in them - 2nd Approach
03:24
Using SQOOP Option files to simplify CLI Commands
05:01
Running SQOOP Import in Debug mode
04:38
Importing & Storing Data in Textual Format on Hadoop
04:18
Importing & Storing Data in AVRO Format on Hadoop
07:13
Importing & Storing Data in SEQUENCE Format on Hadoop
03:07
Importing & Storing Data in PARQUET Format on Hadoop
03:16
Compressing Imported Data
04:50
Running Custom MySQL Queries on Source Tables
05:48
Handling NULL values in Source Dataset
03:46
Setting Custom Field Separators in Imported Data
02:55
Handling Escape Characters while Importing
10:39
Avoid Enclosing all Data Values while Importing
04:14
Incremental Loading of Delta data while Importing - Part 1
08:19
Incremental Loading of Delta data while Importing - Part 2
04:51
Importing Data Directly into Hive Table
06:08
Using HCATALOG to Load Data in ORC File Format
08:26
Load ALL tables from MySQL to Hadoop
05:47
Load ALL tables from MySQL to Hive Database
06:29
+ Apache SQOOP - EXPORT
4 lectures 35:58
Export a Hive table to MySQL table
13:33
Export Specific Columns from Hive table to a MySQL table
07:52
Avoid Partial Data Exports in SQOOP
10:00
When Update Record is OK in SQOOP Export
04:33
+ Apache SQOOP - JOBS
2 lectures 16:04
SQOOP Jobs - Create, List, Show, Execute & Delete Operations
12:52
Make SQOOP job remember MySQL Database Password For Subsequent executions
03:12
+ Conclusion
2 lectures 01:44
What Next
01:18
Continue Learning
00:26