Basic knowledge of Distributed data processing architecture.
Basic knowledge of Python would be helpful.
Apache Beam is a unified and portable programming model for both Batch and Streaming use cases.
Earlier we could run Spark, Flink & Cloud Dataflow Jobs only on their respective clusters. But now Apache Beam has come up with a portable programming model where we can build language agnostic Big data pipelines and run it using any Big data engine (Apache Spark, Flink or in Google Cloud Platform using its Cloud Dataflow and many more Big data engines).
Apache Beam is the future of building Big data processing pipelines and is going to be accepted by mass companies due to its portability. Many big companies have even started deploying Beam pipelines in their production servers.
What's included in the course ?
Complete Apache Beam concepts explained from Scratch to Real-Time implementation.
Each and every Apache Beam concept is explained with aHANDS-ON example of it.
Include even those concepts, the explanation to which is not very clear even in Apache Beam's official documentation.
Build 2 Real-time Big data case studies using Beam.
Load data to Google BigQuery Tables from Beam pipeline.
Codes and Datasets used in lectures are attached in the course for your convenience.
Who this course is for:
Students who want to learn Apache Beam from scratch to its Live Project Implementation.
Data engineers who want to build unified & portable Big data processing pipelines.
Developers who want to learn a futuristic programming model for Big data processing.
14 sections • 63 lectures • 5h 24m total length
Architecture of Apache Beam
Flow of Beam's Programming Model
Basic Terminologies in Beam
Structure of a Beam Pipeline
Various Read Transforms in Beam
Various Write Transforms in Beam
Map, FlatMap & Filter - Part 2
Create Composite Transforms
CoGroupBy for Joins
How to access files from Google Drive
Requirements and Data walk-through for Card skippers
Identifying Credit card payment skippers
Requirements and Data walk-through for Loan Deafulters
At Hadoop Real time Learning, the courses are made keeping in mind the Real-time implementation of Big data technologies in Live Projects. We make courses which majorly consist of Hands-On & Practicals. All our courses contain a detailed knowledge of a technology from Scratch to Advance level. Course's lectures explain the codes in such a way that even a Non-technical person can understand.