IT & Software Other IT & Software Python

Python and Spark - Setup Development Environment

Setup Development Environment for Python and Spark on Windows 10
Rating: 4.0 out of 5 (412 ratings)
22,523 students
Created by Durga Viswanatha Raju Gadiraju, Itversity Support
English
Setup environment to explore and learn Python and Spark

Requirements

  • Basic computer skills
  • At least 4 GB RAM
  • 64 bit operating system (preferably Windows 10)

Description

Challenges to setup development environment using Windows

  • Require external tools such as 7z to uncompress and untar
  • Understanding about environment variables
  • Compatibility issues with Windows internals (winutils)
  • File paths with special characters
  • and more

It is very important to set up development environment addressing all the challenges so that we can spend time in development. Most of the above issues is road blocks for beginners.

Hence we created brief course covering all the important aspects of setting up development environment for Python and Spark.

Once you complete this course, you can proceed with other courses which will cover Spark with Python.

Who this course is for:

  • Any one who want to learn Python and then Spark

Course content

1 section • 13 lectures • 46m total length

  • Introduction - Setup Python, PyCharm and Spark on Windows
    00:31
  • Understanding Pre-requisites
    04:12
  • Setup Python on Windows 10
    03:07
  • Configure Environment Variables for Python
    02:31
  • Setup Pycharm on Windows 10
    05:28
  • Develop Python application using PyCharm with Run time Parameters
    02:31
  • Download Spark - Compressed tar ball
    01:38
  • Install 7z to uncompress and untar compressed tar balls
    01:00
  • Setup Spark using downloaded compressed tar ball
    02:26
  • Install JDK and JRE on Windows 10
    06:05
  • Configure environment variables for Spark
    03:46
  • Setup Winutils for integration of HDFS APIs and Windows file system
    06:30
  • Develop simple Pyspark application using PyCharm
    07:06

Featured review

Dibyajyoti Sarkar
Dibyajyoti Sarkar
54 courses
6 reviews
Rating: 5.0 out of 5a year ago
I found this course to be very helpful. It does a great job of explaining how to set up Python and Spark on windows. Even though the videos demonstrate the installation of Python 2.7 and Spark 1.6.; using the same procedures I could easily set up Python 3.x and Spark 2.x on my system. Thank you again for this helpful course!

Instructors

Durga Viswanatha Raju Gadiraju
Technology Adviser and Evangelist
Durga Viswanatha Raju Gadiraju
  • 4.2 Instructor Rating
  • 9,010 Reviews
  • 166,681 Students
  • 18 Courses

13+ years of experience in executing complex projects using vast array of technologies including Big Data and Cloud.

ITVersity, Inc. - a US based organization to provide quality training for IT professionals and we have the track record of training hundreds of thousands of professionals globally.

Building IT career for people with required tools such as high quality material, labs, live support etc to upskill and cross skill is paramount for our organization.

At this time our training offerings are focused on following areas:

* Application Development using Python and SQL

* Big Data and Business Intelligence

* Cloud

* Datawarehousing, Databases

Itversity Support
Support Account for ITVersity Courses.
Itversity Support
  • 4.2 Instructor Rating
  • 8,369 Reviews
  • 150,911 Students
  • 15 Courses

We have built a team to support going forward. If you send messages to this account for our courses, they will be sent to our Helpdesk from where we will be rewriting to our team.