Setup Big Data Development Environment

Setup Big Data Development Environment for free on Mac or Windows
Free tutorial
Rating: 3.8 out of 5 (364 ratings)
27,164 students
Setup Big Data Development Environment
Free tutorial
Rating: 3.8 out of 5 (364 ratings)
27,164 students
Understand how to setup development environment to learn big data technologies.

Requirements

  • Students need to have modern laptop with 64 bit OS and at least 16 GB RAM
Description

Big Data is open source and there are many technologies one need to learn to be proficient in Big Data eco system tools such as Hadoop, Spark, Hive, Pig, Sqoop etc. This course will cover how to set up development environment on personal computer or laptop using distributions such as Cloudera or Hortonworks. Both Cloudera and Hortonworks provide virtual machine image which contain all Big Data eco system tools packaged. This free course will provide 

  • Comparison of Virtualization software such as Virtualbox and VMWare
  • Step by step instructions to set up virtualization software such as virtualbox or VMWare
  • Choosing Cloudera or Hortonworks image
  • Step by step instructions to set up VM using chosen image
  • Setup necessary additional components such as MySQL database and log generation tool
  • Review HDFS, Map Reduce, Sqoop, Pig, Hive, Spark etc
Who this course is for:
  • Any one who want to learn multiple technologies in Big Data eco system. They need to have basic programming skills.
Course content
8 sections • 51 lectures • 6h 19m total length
  • Getting Started
    04:26
  • Overview of Big Data sandboxes or virtual machine images
    05:11
  • Pre-requisites
    03:24
  • Choosing Virtualization Software (very important)
    05:52
  • Installing VMWare Fusion on Mac
    03:34
  • Installing Oracle VirtualBox on Mac
    04:10
  • Setup Cloudera Quickstart VM - VMWare image
    10:16
  • Review retail_db and gen_logs in Cloudera Quickstart VM
    12:08
  • Download Cloudera Quickstart VM for Virtualbox
    03:35
  • Setup Cloudera Quickstart VM for Virtualbox
    17:55
  • Review retail_db and gen_logs in Cloudera Quickstart VM
    12:08
  • Setup Hortonworks Sandbox on VMWare - Mac
    12:32
  • Setup MySQL Database - retail_db
    10:08
  • Setup gen_logs application to generate logs
    06:12
  • Setup Hortonworks Sandbox on Virtual Box
    12:33
  • Reset admin password
    04:43
  • Setup MySQL Database - retail_db
    10:08
  • Setup gen_logs application to generate logs
    06:12
  • Setup Eclipse with Maven Plugin - Introduction
    02:11
  • Setup Eclipse with Maven Plugin
    08:07
  • Create java application using Maven Project
    08:49
  • Develop word count program introduction
    02:17
  • Develop word count program
    11:33
  • Run word count program
    07:44
  • Setup github project - Introduction
    05:59
  • Download and setup github project
    12:36
  • Validate github project
    07:06
  • Setup scala and sbt - Introduction
    04:28
  • Setup and Validate Scala
    14:19
  • Run simple scala application
    05:50
  • Setup sbt and run scala application
    10:54
  • Setup Scala IDE for Eclipse - Introduction
    03:18
  • Install Scala IDE for Eclipse
    11:00
  • Integrate sbt with Scala IDE for Eclipse
    17:56
  • Develop Spark applications using Scala IDE - Introduction
    01:37
  • Develop Spark applications using Scala IDE and sbt
    15:16
  • Run Spark applications on cluster
    16:23
  • Introduction
    03:06
  • Setup Java and JDK
    05:16
  • Install Scala with IntelliJ IDE
    06:53
  • Develop Hello World Program using Scala
    09:07
  • Setup sbt and run application HelloWorld
    04:18
  • Add spark dependencies to the application
    04:31
  • Setting up winutils.exe on Windows (64 bit)
    04:37
  • Setup Data Sets - retail_db
    03:16
  • Develop first spark application - Get revenue for each order from order_items
    07:46
  • Build Jar file using sbt
    02:07
  • Download and install Spark using 7z on Windows
    04:07
  • Configure environment variables for Spark on Windows
    02:12
  • Running spark job using spark-shell
    03:03
  • Validating spark job from jar file using spark-submit
    06:15

Instructors
Technology Adviser and Evangelist
Durga Viswanatha Raju Gadiraju
  • 4.2 Instructor Rating
  • 8,739 Reviews
  • 149,340 Students
  • 19 Courses

13+ years of experience in executing complex projects using vast array of technologies including Big Data and Cloud.

I found itversity, llc - a US based startup to provide quality training for IT professionals and staffing as well as consulting solutions for enterprise clients. I have trained thousands of IT professionals in vast array of technologies including Big Data and Cloud.

Building IT career for people and provide quality services to the clients will be paramount to our organization.

As an entry strategy itversity will be providing quality training in the areas of ABCD

* Application Development
* Big Data and Business Intelligence
* Cloud
* Datawarehousing, Databases

Support Account for ITVersity Courses.
Itversity Support
  • 4.2 Instructor Rating
  • 8,739 Reviews
  • 148,213 Students
  • 19 Courses

We have built a team to support going forward. If you send messages to this account for our courses, they will be sent to our Helpdesk from where we will be rewriting to our team.