Setup Big Data Development Environment

Setup Big Data Development Environment for free on Mac or Windows
Free tutorial
Rating: 4.3 out of 5 (351 ratings)
26,536 students
Setup Big Data Development Environment
Free tutorial
Rating: 4.3 out of 5 (351 ratings)
26,536 students
Understand how to setup development environment to learn big data technologies.

Requirements

  • Students need to have modern laptop with 64 bit OS and at least 16 GB RAM
Description

Big Data is open source and there are many technologies one need to learn to be proficient in Big Data eco system tools such as Hadoop, Spark, Hive, Pig, Sqoop etc. This course will cover how to set up development environment on personal computer or laptop using distributions such as Cloudera or Hortonworks. Both Cloudera and Hortonworks provide virtual machine image which contain all Big Data eco system tools packaged. This free course will provide 

  • Comparison of Virtualization software such as Virtualbox and VMWare
  • Step by step instructions to set up virtualization software such as virtualbox or VMWare
  • Choosing Cloudera or Hortonworks image
  • Step by step instructions to set up VM using chosen image
  • Setup necessary additional components such as MySQL database and log generation tool
  • Review HDFS, Map Reduce, Sqoop, Pig, Hive, Spark etc
Who this course is for:
  • Any one who want to learn multiple technologies in Big Data eco system. They need to have basic programming skills.
Curriculum
8 sections • 51 lectures • 6h 19m total length
  • Getting Started
  • Overview of Big Data sandboxes or virtual machine images
  • Pre-requisites
  • Choosing Virtualization Software (very important)
  • Installing VMWare Fusion on Mac
  • Installing Oracle VirtualBox on Mac
  • Setup Cloudera Quickstart VM - VMWare image
  • Review retail_db and gen_logs in Cloudera Quickstart VM
  • Download Cloudera Quickstart VM for Virtualbox
  • Setup Cloudera Quickstart VM for Virtualbox
  • Review retail_db and gen_logs in Cloudera Quickstart VM
  • Setup Hortonworks Sandbox on VMWare - Mac
  • Setup MySQL Database - retail_db
  • Setup gen_logs application to generate logs
  • Setup Hortonworks Sandbox on Virtual Box
  • Reset admin password
  • Setup MySQL Database - retail_db
  • Setup gen_logs application to generate logs
  • Setup Eclipse with Maven Plugin - Introduction
  • Setup Eclipse with Maven Plugin
  • Create java application using Maven Project
  • Develop word count program introduction
  • Develop word count program
  • Run word count program
  • Setup github project - Introduction
  • Download and setup github project
  • Validate github project
  • Setup scala and sbt - Introduction
  • Setup and Validate Scala
  • Run simple scala application
  • Setup sbt and run scala application
  • Setup Scala IDE for Eclipse - Introduction
  • Install Scala IDE for Eclipse
  • Integrate sbt with Scala IDE for Eclipse
  • Develop Spark applications using Scala IDE - Introduction
  • Develop Spark applications using Scala IDE and sbt
  • Run Spark applications on cluster
  • Introduction
  • Setup Java and JDK
  • Install Scala with IntelliJ IDE
  • Develop Hello World Program using Scala
  • Setup sbt and run application HelloWorld
  • Add spark dependencies to the application
  • Setting up winutils.exe on Windows (64 bit)
  • Setup Data Sets - retail_db
  • Develop first spark application - Get revenue for each order from order_items
  • Build Jar file using sbt
  • Download and install Spark using 7z on Windows
  • Configure environment variables for Spark on Windows
  • Running spark job using spark-shell
  • Validating spark job from jar file using spark-submit

Instructors
Technology Adviser and Evangelist
Durga Viswanatha Raju Gadiraju
  • 4.1 Instructor Rating
  • 8,037 Reviews
  • 140,918 Students
  • 19 Courses

13+ years of experience in executing complex projects using vast array of technologies including Big Data and Cloud.

I found itversity, llc - a US based startup to provide quality training for IT professionals and staffing as well as consulting solutions for enterprise clients. I have trained thousands of IT professionals in vast array of technologies including Big Data and Cloud.

Building IT career for people and provide quality services to the clients will be paramount to our organization.

As an entry strategy itversity will be providing quality training in the areas of ABCD

* Application Development
* Big Data and Business Intelligence
* Cloud
* Datawarehousing, Databases

Support Account for ITVersity Courses.
Itversity Support
  • 4.1 Instructor Rating
  • 7,783 Reviews
  • 138,836 Students
  • 18 Courses

We have built a team to support going forward. If you send messages to this account for our courses, they will be sent to our Helpdesk from where we will be rewriting to our team.