Learn By Example : Apache Storm

Name: Learn By Example : Apache Storm
Rating: 4.1 (542 reviews)

25 Solved examples on Real Time Stream Processing

Created byLoony Corn

Last updated 2/2017

English

English [Auto],

What you'll learn

Build a Storm Topology for processing data
Manage reliability and fault tolerance of the topology
Control parallelism using different grouping strategies
Perform complex transformations using Trident
Apply Machine Learning algorithms on the fly in Storm applications

Course content

12 sections • 35 lectures • 4h 4m total length

You, This Course and Us2:06
We'll start with an introduction, what the course covers and who it benefits.

How does Twitter compute Trends?5:42
Understand how stream processing is different from batch processing
Improving Performance using Distributed Processing5:39
Stream processing is great for certain applications, but performance can be an issue at large scale. How do we solve this?
Building blocks of Storm Topologies5:38
Understand Spouts and Bolts which make up a Storm topology
Adding Parallelism in a Storm Topology4:54
Understand how a Storm topology allows parellelism across components
Components of a Storm Cluster4:06
A Storm topology runs on a cluster. Understand the different services which run on the cluster

A Simple Hello World Topology4:13
Build a simple hello world topology in storm with a spout that auto generates integers and a bolt that multiplies each by two, run on a local cluster.
Ex 1: Implementing a Spout11:10
Ex 1: Implementing a Bolt4:43
Ex 1: Submitting the Topology5:14
Build a topology with a spout and bolt using topology builder, configure debug true, and submit to a local cluster. Observe spout outputs doubled to even numbers.

Ex 6 : Shuffle Grouping6:42
Ex 7: Fields Grouping4:37
Explore field grouping in Apache Storm by routing data with a bucket field to the same bolt, enabling consistent aggregation such as word counts.
Ex 8: All Grouping2:22
Ex 9: Custom Grouping5:16
Ex 10: Direct Grouping5:39
Build a topology using direct grouping to route data directly to specific bolts, embedding the grouping logic in the component and evenly distributing buckets among bolts.

Requirements

Experience in Java programming and familiarity with using Java frameworks
A Java IDE such as IntelliJ Idea should be installed

Description

Storm is to real-time stream processing what Hadoop is to batch processing. Using Storm you can build applications which need you to be highly responsive to the latest data and react within seconds and minutes, such as finding the latest trending topics on twitter, or monitoring spikes in payment gateway failures. From simple data transformations to applying machine learning algorithms on the fly, Storm can do it all.

This course has 25 Solved Examples on building Storm Applications.

What's covered?

1) Understanding Spouts and Bolts which are the building blocks of every Storm topology.

2) Running a Storm topology in the local mode and in the remote mode

3) Parallelizing data processing within a topology using different grouping strategies : Shuffle grouping, fields grouping, Direct grouping, All grouping, Custom Grouping

4) Managing reliability and fault-tolerance within Spouts and Bolts

5) Performing complex transformations on the fly using the Trident topology : Map, Filter, Windowing and Partitioning operations

6) Applying ML algorithms on the fly using libraries like Trident-ML and Storm-R.

Who this course is for:

Yep! Engineers looking to set up end-to-end data processing pipelines that react to changes in real time
Yep! Folks familiar with Batch processing technologies like Hadoop who want to learn more about Stream processing

Learn By Example : Apache Storm

What you'll learn

Explore related topics

Course content

Start Here1 lecture • 2min

Stream Processing with Storm5 lectures • 26min

Implementing a Hello World Topology4 lectures • 25min

Processing Data using Files4 lectures • 34min

Running a Topology in the Remote Mode2 lectures • 15min

Adding Parallelism to a Storm Topology5 lectures • 25min

Section 7: Building a Word Count Topology1 lecture • 10min

Remote Procedure Calls Using Storm1 lecture • 13min

Managing Reliability of Topologies1 lecture • 11min

Integrating Storm with Different Sources/Sinks2 lectures • 16min

Requirements

Description

Who this course is for: