Python and Spark - Setup Development Environment
- Basic computer skills
- At least 4 GB RAM
- 64 bit operating system (preferably Windows 10)
Challenges to setup development environment using Windows
- Require external tools such as 7z to uncompress and untar
- Understanding about environment variables
- Compatibility issues with Windows internals (winutils)
- File paths with special characters
- and more
It is very important to set up development environment addressing all the challenges so that we can spend time in development. Most of the above issues is road blocks for beginners.
Hence we created brief course covering all the important aspects of setting up development environment for Python and Spark.
Once you complete this course, you can proceed with other courses which will cover Spark with Python.
Who this course is for:
- Any one who want to learn Python and then Spark
- 00:31Introduction - Setup Python, PyCharm and Spark on Windows
- 04:12Understanding Pre-requisites
- 03:07Setup Python on Windows 10
- 02:31Configure Environment Variables for Python
- 05:28Setup Pycharm on Windows 10
- 02:31Develop Python application using PyCharm with Run time Parameters
- 01:38Download Spark - Compressed tar ball
- 01:00Install 7z to uncompress and untar compressed tar balls
- 02:26Setup Spark using downloaded compressed tar ball
- 06:05Install JDK and JRE on Windows 10
- 03:46Configure environment variables for Spark
- 06:30Setup Winutils for integration of HDFS APIs and Windows file system
- 07:06Develop simple Pyspark application using PyCharm
13+ years of experience in executing complex projects using vast array of technologies including Big Data and Cloud.
ITVersity, Inc. - a US based organization to provide quality training for IT professionals and we have the track record of training hundreds of thousands of professionals globally.
Building IT career for people with required tools such as high quality material, labs, live support etc to upskill and cross skill is paramount for our organization.
At this time our training offerings are focused on following areas:
* Application Development using Python and SQL
* Big Data and Business Intelligence
* Datawarehousing, Databases