Real Time Streaming using Apache Spark Streaming
5.0 (1 rating)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
11 students enrolled
Wishlisted Wishlist

Please confirm that you want to add Real Time Streaming using Apache Spark Streaming to your Wishlist.

Add to Wishlist

Real Time Streaming using Apache Spark Streaming

Analyze data in real-time using the Apache Spark Streaming API
5.0 (1 rating)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
11 students enrolled
Created by Packt Publishing
Last updated 7/2017
English
Curiosity Sale
Current price: $10 Original price: $125 Discount: 92% off
30-Day Money-Back Guarantee
Includes:
  • 1 hour on-demand video
  • 1 Supplemental Resource
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
What Will I Learn?
  • Implement stream processing using Apache Spark Streaming
  • Consume events from the source (for instance, Kafka), apply logic on it, and send it to a data sink.
  • Understand how to deduplicate events when you have a system that ensures at-least-once deliver.
  • Learn to tackle common stream processing problems.
  • Create a job to analyze data in real time using the Apache Spark Streaming API.
  • Master event time and processing time
  • Single event processing and the micro-batch approach to processing events
  • Learn to sort infinite event streams
View Curriculum
Requirements
  • A basic understanding and functional knowledge of Apache Spark, stream processing, and big data are required.
Description

Spark is the technology that allows us to perform big data processing in the MapReduce paradigm very rapidly, due to performing the processing in memory without the need for extensive I/O operations.

Recently, the streaming approach to processing events in near real time became more widely adopted and more necessary. In this course, you will learn how to handle big amount of unbounded infinite streams of data. You will analyze data and draw conclusions from it. Furthermore, we will look at common problems when processing event streams: sorting, watermarks, deduplication, and keeping state (for example, user sessions). You will also implement streaming processing using Spark Streaming and analyze traffic on a web page in real time.

About the Author :

Tomasz Lelek is a Software Engineer, programming mostly in Java, Scala. He is a fan of microservices architecture, and functional programming. He has dedicated considerable time and effort to be better every day. He recently dived into Big Data technologies such as Apache Spark and Hadoop. Tomasz is passionate about nearly everything associated with software development.Recently he was a speaker at conferences in Poland - Confitura and JDD (Java Developers Day) and also at Krakow Scala User Group. He has also conducted a live coding session at Geecon Conference.

Who is the target audience?
  • The course is for software engineers interested in big data processing.
Students Who Viewed This Course Also Viewed
Curriculum For This Course
+
Understanding a Spark Streaming
5 Lectures 28:26
This video provides an overview of the entire course
Preview 02:19

In this video, we will explore the Spark-Streaming Architecture and API

Introduction to Spark Streaming API
09:13

In this video, we will see how to create a project in Spark streaming

Creating a Project in Spark Streaming
05:26

In this video, we will define data Source and data sink
Preview 07:58

In this video, we will see how to implement testing

Creating Base for Testing Spark Streaming
03:30
+
Implementing Stream Processing
4 Lectures 16:53

In this video, we will handle Unbounded Data.

Preview 02:57

Using Event Time and Processing Time
02:49

In this video, we will sort a stream of data.
Sorting Stream Data
05:15

In this video, we will deduplicate our events.
Deduplicating Data
05:52
+
Implementing Transformations and Processing Logic
4 Lectures 14:14
In this video, we will implement transformation and actual logic of our processing.
Implementing Job Processing Logic
02:49

In this video, we will write test for the streaming job.
Writing Test for Steaming Job
03:05

In this video, we will create processing logic that needs to keep state of the user session.
Creating Processing Logic That Needs to Keep State of the User Session
04:42

In this video, we will summarize all the topics covered in this course.

Summary of Stream Processing
03:38
About the Instructor
Packt Publishing
3.9 Average rating
7,336 Reviews
52,354 Students
616 Courses
Tech Knowledge in Motion

Packt has been committed to developer learning since 2004. A lot has changed in software since then - but Packt has remained responsive to these changes, continuing to look forward at the trends and tools defining the way we work and live. And how to put them to work.

With an extensive library of content - more than 4000 books and video courses -Packt's mission is to help developers stay relevant in a rapidly changing world. From new web frameworks and programming languages, to cutting edge data analytics, and DevOps, Packt takes software professionals in every field to what's important to them now.

From skills that will help you to develop and future proof your career to immediate solutions to every day tech challenges, Packt is a go-to resource to make you a better, smarter developer.

Packt Udemy courses continue this tradition, bringing you comprehensive yet concise video courses straight from the experts.