Big Data Internship Program - Data Processing - Hive and Pig

Name: Big Data Internship Program - Data Processing - Hive and Pig
Rating: 4.3 (39 reviews)

Provide higher-level language to facilitate large-data processing.

Created byBig Data Trunk

Last updated 1/2017

English

What you'll learn

Have excellent understanding of Apache Hive and Pig tool with hands-on experience .
Understand the working of a project in real-world scenario.
Work experience in end-to-end Project ( Data Masking) and can mention in Resume .

Course content

5 sections • 34 lectures • 2h 13m total length

Introduction to the course2:04
In This video, we have explained the course structure of course, How our course is useful for Big data experts and beginners.
Introduction to Data processing5:23
In This video, we have explained what is data processing, how data processing is done in big data environment, what is big data cycle. Why big data processing is important in different Areas.

Hadoop in Retail Industry2:58
In this video, we have explained how Hadoop is applicable in the retail market, how hadoop can play important role in customer analysis.
Hive Introduction3:24
In this video, we have given a small introduction of the hive, why
Facebook uses hive, where the hive was developed, what are hive
features.
Hive Introduction
Hive Architecture4:26
In this video, we have explained what is hive Architecture, how the hive is integrated with other hive components, how hive executes hive query.
Hive Architecture
DataTypes in Hive3:07
In this video we have explained, what are the data type available in
HiveQL, what are primitive data type available, Collection data types
etc
Managed table and External table in Hive1:51
In this video, we have explained what is an internal table, what is an
external table, How they are used, what is the feature of hive internal
table and external table.
Manage and external tables in hive
Demonstrating difference between Internal and external table8:00
In this video, we have explained what is an external table and internal
table, what is the benefit of an external table over an internal table.
Demonastrating diffrent between external and interrnal
Hands on Lab Hive External Vs InternalTable1:06
Partitions in Hive with demo8:25
In this video we have explained partitioning in the hive, what is the meaning of partitioning table.
Hands on Lab Hive Partition0:33
Partition in hive
HIve Dynamic Partitioning3:42
In this video we have explained Dynamic partition in hive, what is the usage of hive dynamic table, how to create dynamic table etc.
Hands on Lab HIve Dynamic Partitioning0:48
Hive dynamic Partition

What is Apache PIG?4:37
In this video, we have given a brief introduction of Pig, why it is used, and what are the characteristic of the pig.
PIG Architecture2:02
In this video, we have shown Pig architecture, how Pig statement is executed in Pig grunt mode.
Pig Data Types3:18
In this video, we have shown what are the different data types available in Pig, Type of data types, what are relations, bag, tuple etc.
Pig Data Types
Pig Latin6:07
In this video, we have shown what is Pig Latin, what are various basic commands which are used in Pig Script, Pig latin Map Reduce, Use of Python with Pig
Pig Latin
PIG Running Modes3:40
In this video, we have described different running mode, and their usage, How pig script is executed in different modes
PIG Running Modes
PIG Operators7:03
In this video we have shown the different type of operators available in Pig Latin, like binary, ternary, flatter, how to load data using PigStorage(), Dump operator, store operator, limit and distinct, order by, grouping etc.
PIG Operators
PIG Wordcount example3:18
In this video, we have shown how we can execute word count task in pig latin.
Pig Wordcount lab0:34

BookRecommendationProject5:13
In this video we have explain how to execute our Book Recommendation Project by using hive, sqoop, mysql. How to upload data in system for processing.
BookRecommendationProject-28:12
In this video we have explained some attribute of table, how we can access them and how we can optimize query execution in hive, we have done some hands-on recommendation database, for analysis of tables and seen the results.

Data Masking Project Overview3:42
In this video, we have explained the data masking project, which components we are going to use, what is the use of data masking etc, what are project requirement, what is a flow of the project.
Data Masking Project Solution Design3:19
In this video, we have explained data masking solution, how this project
is executed, what is the goal of the project, what Softwares/tools we
are going to use in execution.
Data Masking Project Solution Walkthrough9:05
In this video we have explain the step-by-stpe flow of Data Masking project, and different stages of project.
DataMaskingProject-step1(Create tables inMySql)5:41
In this video we have explained how to create table in mysql, how to load data in mysql from file.
Hands on Lab DataMaskingProject-step1-Document1:48
Step2:Creating and importing data in hive external tables3:40
In this video, we have explained how to create an external table, and
how to load data in the external table, how to import data from external
table to hive table.
Hands on Lab DataMaskingProject-step2- Document0:55
Step3:Creating UDFs in Java6:36
In this video we have explained, how to crreate UDFs, and how to jar for Data Masking Project
Hands on Lab DataMaskingProject-step3(udf)- Document2:20
Step4:Exporting data to MySql in masked database5:34
In this video, we have shown you actual execution of data masking, by using MySQL, sqoop
Hands on Lab DataMaskingProjectStep4 - Document1:19

Requirements

Should know the basics of BigData concepts like-HDFS,MapReduce and some knowledge of RDBMS.
Should take our Part-1 free course (Big data Internship Program - Foundation) to understand these concepts better. (Not mandatory but desirable).
Should take our Part-1 free course (Big data Internship Program - ingestion ) to understand sqoop and flume. ( For our Book Recommendation Project only).

Description

This course is part of “Big data Internship Program” which is aligned to a typical Big data project life cycle stage.

Foundation
Ingestion
Storage
Processing
Visualization

This course is focused on Data Processing in Big data.This course is suitable for developers, data analysts and business analysts. Experience with SQL and scripting languages is recommended, but is not required.

You will learn

Understanding of Hive core concept and architecture.
How to create and manipulate tables using Hive.
Advanced features of Hive.
Hive Best Practices
Performing real-time, complex queries on datasets
Pig’s Architecture
Reading and Writing Data with Pig
Pig Best Practices

Project work -

Provide Data in Hive and manipulate the data for Our Book Recommendation project.
One Ad-on project -- Data Masking with hive and sqoop

Who this course is for:

This course is for anyone who wants to learn about Hive and Pig in details with hand exprience.
Students who want to do internship.
Big Data Analytics Professional.
Developers who want to get clear concept .

Big Data Internship Program - Data Processing - Hive and Pig

What you'll learn

Explore related topics

Course content

Data Processing Introduction in Big Data2 lectures • 7min

HIve11 lectures • 38min

Pig8 lectures • 31min

Data Processing in Recommendation Project2 lectures • 13min

Ad-on Project Data Masking11 lectures • 44min

Requirements

Description

Who this course is for: