Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Development Tools No-Code Development
Business
Entrepreneurship Communications Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certification Network & Security Hardware Operating Systems Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design Design Thinking 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition Yoga Mental Health Dieting Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Teacher Training Test Prep Other Teaching & Academics
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Cisco CCNA Amazon AWS CompTIA Security+ Microsoft AZ-900
Graphic Design Photoshop Adobe Illustrator Drawing Digital Painting InDesign Character Design Canva Figure Drawing
Life Coach Training Neuro-Linguistic Programming Personal Development Mindfulness Personal Transformation Life Purpose Meditation CBT Emotional Intelligence
Web Development JavaScript React CSS Angular PHP Node.Js WordPress Vue JS
Google Flutter Android Development iOS Development React Native Swift Dart Programming Language Mobile Development Kotlin SwiftUI
Digital Marketing Google Ads (Adwords) Social Media Marketing Google Ads (AdWords) Certification Marketing Strategy Internet Marketing YouTube Marketing Email Marketing Retargeting
Microsoft Power BI SQL Tableau Business Analysis Data Modeling Business Intelligence MySQL Data Analysis Blockchain
Business Fundamentals Entrepreneurship Fundamentals Business Strategy Business Plan Startup Online Business Freelancing Blogging Home Business
Unity Game Development Fundamentals Unreal Engine C# 3D Game Development C++ 2D Game Development Unreal Engine Blueprints Blender
30-Day Money-Back Guarantee
Development Database Design & Development Apache Spark

Big Data Processing with Apache Spark

Efficiently tackle large data sets and big data analysis challenges using Spark and Python
Rating: 4.3 out of 54.3 (4 ratings)
48 students
Created by Packt Publishing
Last updated 1/2019
English
English [Auto]
30-Day Money-Back Guarantee

What you'll learn

  • Write your own Python programs that can interact with Spark
  • Implement data stream consumption using Apache Spark
  • Recognize common operations in Spark to process known data streams
  • Integrate Spark streaming with Amazon Web Services
  • Create a collaborative filtering model with Python and the movielens dataset
  • Apply processed data streams to Spark machine learning APIs

Course content

4 sections • 27 lectures • 3h 29m total length

  • Preview02:33
  • Preview04:50
  • Lesson Overview
    03:35
  • Introduction to Spark and Resilient Distributed Datasets
    16:14
  • Operations Supported by the RDD API
    15:41
  • Map Reduce Operations
    07:28
  • Self-Contained Python Spark Programs
    10:47
  • Nested Functions and Standalone Python Programs
    10:10
  • Introduction to SQL, Datasets, and DataFrames
    14:16
  • Lesson Summary
    00:43
  • Test Your Knowledge
    3 questions

  • Preview01:17
  • Introduction to Streaming Architectures
    02:28
  • Introduction to Discretized Streams (Dstreams)
    12:27
  • Operations Supported by the Spark Streaming API
    17:56
  • Windowing Operations
    12:59
  • Structured Streaming
    11:08
  • Lesson Summary
    00:40
  • Test Your Knowledge
    5 questions

  • Preview01:04
  • Spark Integration with AWS Services
    09:55
  • Integrating AWS Kinesis and Python
    14:45
  • AWS S3 Basic Functionality
    08:38
  • Kinesis Streams and Spark Streams
    01:53
  • Lesson Summary
    00:39
  • Test Your Knowledge
    4 questions

  • Preview01:12
  • Spark Integration with Machine Learning
    17:44
  • Spark Streaming Windowing Operations
    07:24
  • Lesson Summary
    01:32
  • Test Your Knowledge
    2 questions

Requirements

  • Prior experience of working with Python is recommended.

Description

Processing big data in real  time is challenging due to scalability, information consistency, and  fault-tolerance. Big Data Processing with Apache Spark teaches you how  to use Spark to make your overall analytical workflow faster and more  efficient. You'll explore all core concepts and tools within the Spark  ecosystem, such as Spark Streaming, the Spark Streaming API, machine  learning extension, and structured streaming.

You'll begin by learning data processing fundamentals using Resilient  Distributed Datasets (RDDs), SQL, Datasets, and Dataframes APIs. After  grasping these fundamentals, you'll move on to using Spark Streaming  APIs to consume data in real time from TCP sockets, and integrate Amazon  Web Services (AWS) for stream consumption.

By the end of this course, you’ll not only have understood how to use  machine learning extensions and structured streams but you’ll also be  able to apply Spark in your own upcoming big data projects. 

About the Author

Manuel Ignacio Franco  Galeano is a computer scientist from Colombia. He works for Fender  Musical Instruments as a lead engineer in Dublin, Ireland. He holds a  master's degree in computer science from University College, Dublin UCD.  His areas of interest and research are music information retrieval,  data analytics, distributed systems, and blockchain technologies.

Nimish Narang has graduated from UBC with a degree in biology and  computer science in 2016. He has developed Mobile apps for Android and  iOS since 2015. He is focused on data analysis and machine learning from  the past two years and has previously published Keras and Professional  Scala with Packt. 

Who this course is for:

  • This course is for you if you are a software engineer, architect, or IT professional who wants to explore distributed systems and big data analytics

Instructor

Packt Publishing
Tech Knowledge in Motion
Packt Publishing
  • 3.9 Instructor Rating
  • 59,001 Reviews
  • 353,963 Students
  • 1,418 Courses

Packt has been committed to developer learning since 2004. A lot has changed in software since then - but Packt has remained responsive to these changes, continuing to look forward at the trends and tools defining the way we work and live. And how to put them to work.

With an extensive library of content - more than 4000 books and video courses -Packt's mission is to help developers stay relevant in a rapidly changing world. From new web frameworks and programming languages, to cutting edge data analytics, and DevOps, Packt takes software professionals in every field to what's important to them now.

From skills that will help you to develop and future proof your career to immediate solutions to every day tech challenges, Packt is a go-to resource to make you a better, smarter developer.

Packt Udemy courses continue this tradition, bringing you comprehensive yet concise video courses straight from the experts.



  • Udemy for Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Impressum Kontakt
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Featured courses
Udemy
© 2021 Udemy, Inc.