Udemy Business

Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

Apache Flink 1.17.x DataStream API Streaming即時計算完全教程

Name: Apache Flink 1.17.x DataStream API Streaming即時計算完全教程
Rating: 4.3 (3 reviews)

Build powerful stream and batch processing application, real-time data warehouse, data lake by Apache Flink (Java,Kafka)

Created byAlexander Wong

Last updated 5/2024

Chinese (Traditional)

Chinese (Traditional) [Auto],

What you'll learn

清楚知道什麼是 Apache Flink
Apache Flink Stream API
Apache Flink Resource Provider and Deployment Mode
Apache Flink 內部架構
Apache Flink Windows and Window Operator
Apache Flink Watermarks and Strategies
Apache Flink distributed state and backend stores
Apache Flink Job Fault Tolerance

Course content

10 sections • 179 lectures • 24h 8m total length

Prerequisites2:00
Course Objective1:54
課程介紹6:31

第一部分(Flink QuickStart)内容概要2:17
Apache Flink的基本介紹6:59
Apache Flink的特色介紹7:21
Apache Flink的使用場景10:34
Apache Flink的架构预览5:20
Flink獨立叢集安裝10:48
Flink叢集的啟動及停止3:46
Flink DataSet API處理離線檔8:52
Flink DataStream API處理離線文件10:11
Flink Runtime Execution Mode2:54
Flink DataStream處理流數據3:24
Flink DataStream API and Java Lambda表达式6:46
使用Job Manager的WebUI提交Flink Job6:02
使用Flink命令行提交Flink Job4:16

第二部分(Flink Job Deployment Mode)內容摘要1:27
Session Mode,Per-Job Mode和Application Mode概述7:45
Flink job Session deploy Mode的優缺點4:46
Flink job Per-Job Deploy Mode的優缺點3:15
Flink job Application Deploy Mode的優缺點5:25
Session Mode Under Standalone Cluster7:31
Application Mode Under Standalone Cluster8:23
Hadoop叢集的安裝與準備3:31
Session Mode Under Yarn Resource Provider10:13
Per-Job Mode Under Yarn Resource Provider9:43
Application Mode Under Yarn Resource Provider7:32
進一步優化Application Mode的作業提交8:19
Flink Job History Server配置以及啟動7:03

第三部分(Apache Flink Runtime Architecture)內容概要2:12
Apache Flink運行時架構與內部組件詳解10:34
Apache Flink Slots,Tasks,SubTasks概念的介紹8:20
Apache Flink Local Environment For Develop8:12
Apache Flink job operator parallelism7:18
Apache Flink operator並行度優先權詳解7:03
Apache Flink operator chaining介紹5:01
什麼情況下會發生Operator chaining4:54
Apache Flink Operator Chaining API and Programming5:21
Apache Flink Shard Slots Group6:09
Apache Flink Job提交執行的流程-Standalone叢集下的Session Deploy Mode7:46
Apache Flink Job提交執行的流程-Yarn叢集下的Per-Job Deploy Mode5:04
Apache Flink Job提交執行的流程-Yarn叢集下的Session Deploy Mode5:50
深入學習 Apache Flink's data flow graphs9:38

第四部分(Apache Flink 1.17.2 DataStream API)內容概要4:49
Source Collector-Java Collections10:46
Source Collector-DataGen8:47
Source Collector-TCP Socket6:39
Source Collector-FileSystem以StreamFormat讀取處理本機文件8:58
Source Collector-FileSystem以BulkFormat讀取處理本機文件9:53
Source Collector-Hdfs File source connector9:58
Source Collector-Mongodb source connector10:24
Source Collector-Kafka source connector consume text data11:48
Source Collector-Kafka source connector consume JSON data6:24
Source Collector-Kafka source connector consume K,V data13:07
Source Collector-Custom RichSourceFunction8:56
Source Collector-Custom RichParallelSourceFunction9:51
Flink Data Types and TypeInformation10:43
Stream Data Transformations2:52
MAP,FILTER,FLATMAP Transformation10:18
KeyBy and KeyedStream7:32
KeyedStream Rolling Aggregation8:31
KeyedStream ReduceFunction8:14
Multistream Merge Operator - UNION5:03
Multistream Merge Operator-CONNECT(fire alarm application-I)13:05
Multistream Merge Operator-CONNECT(fire alarm application-II)3:14
Multistream Connect Operator-DataStreams Inner Join14:14
Split Data Stream-Side Output Stream10:04
RichFunction and Handling Application Parameters14:04
Data Exchange Strategies and Algorithms5:03
Forward Partitioner Algorithm7:37
Shuffle Partitioner Algorithm3:38
Rebalance Partitioner Algorithm4:23
Broadcast Partitioner Algorithm2:26
Global Partitioner Algorithm4:29
KeyGroupStream Partitioner Algorithm4:07
Rescale Partitioner Algorithm3:37
Custom Partitioner Algorithm3:21
How to apply Accumulators & Counters7:50
Sink Collector-Socket Sink Connector4:19
Sink Collector-LocalFileSystem Sink Connector(TXT)11:10
Sink Collector-LocalFileSystem Sink Connector(AVRO)13:47
Sink Collector-LocalFileSystem Sink Connector(Parquet)3:19
Sink Collector-Hdfs FileSystem Sink Connector(Avro)3:19
Sink Collector-JDBC Sink Connector11:31
Sink Collector-Kafka Sink Connector(Value Only)7:49
Sink Collector-Kafka Sink Connector(Key and Value)7:19
Sink Collector-Kafka Sink Connector(JSON)5:50
Sink Collector-Mongodb Sink Connector9:04
自定义sink collector(old API)9:18
自定义sink collector(new API)6:34

第五部分(Apache Flink 1.17.2 Window API)內容概要3:35
Keyed Window和No-Keyed Window之間的差異10:08
深入解析No-Keyed Window實作原理3:24
深入解析Flink Keyed Window的定義以及源碼分析12:38
Flink 支援的幾種Window類別5:10
Flink Window的生命週期5:58
如何建立Tumbling Process TimeWindow並建置Streaming Application-I14:05
如何建立Tumbling Process TimeWindow並建置Streaming Application-II3:45
深入Flink原始碼進一步分析Window的運算與輸出邏輯12:19
Sliding Process TimeWindow詳解10:00
Session Process TimeWindow詳解10:38
Global TimeWindow詳解9:10
Count Window詳解7:11
WindowedStream Rolling Aggregation Function詳解5:43
WindowedStream reduce operator and ReduceFunction詳解7:04
WindowedStream aggregate operator and AggregateFunction詳解14:29
WindowedStream process operator and ProcessWindowFunction詳解11:40
WindowedStream Side Output詳解8:09
WindowedStream window function結合incrementally function(最佳實務)6:33
WindowedStream apply operator and WindowFunction(Legacy)詳解3:41
WindowedStream Trigger& Trigger API & TriggerResult 詳細講解10:32
WindowedStream 深入Trigger源碼剖析工作原理-I10:35
WindowedStream 深入Trigger源碼剖析工作原理-II15:18
WindowedStream 深入Trigger源碼剖析工作原理-III11:00
WindowedStream如何自定义以及應用Window Trigger-I18:29
WindowedStream如何自定义以及應用Window Trigger-II2:34
WindowedStream Evictor 詳解以及自定义Evictor7:41
Timer and TimerService介紹10:48
深入Timer和TimerService的工作內幕7:07

第六部分(Apache Flink 1.17.2 Watermark)內容概要2:10
Time Semantics Introduce5:46
Process Time and Event Time Introduce9:47
什麼是Flink Watermark(Watermark需要解決哪些問題)10:37
Watermark的生成演算法演進講解11:43
透過程式再現亂序數據和遲到數據9:47
在視窗計算中如何處理亂序數據和遲到數據11:01
Flink Watermark Strategy詳細解說8:58
深入分析Flink Watermark Strategy源碼13:28
自定义Flink Watermark Generator11:47
多並行度下的watermark傳遞詳細講解11:26
透過閱讀原始碼進一步深入分析Flink中Watermark傳遞機制與原理9:23
如何處理空閒source的watermark傳遞8:59
WindowedStream Allowed Lateness詳細解說10:01
WindowedStream透過測輸出流獲取遲到資料避免遺失8:21
JoinedStreams window join詳細講解10:46
KeyedStream interval join詳細講解10:14

第七部分(Apache Flink 1.17.2 Stateful operators and Application)內容概要3:15
Stateful Operator and Stateful Application8:00
Flink中关于State的分类10:13
Flink Keyed State之ValueState使用詳細講解8:34
Flink Keyed State之ReducingState使用詳細講解4:20
Flink Keyed State之AggregatingState使用詳細講解6:11
Flink Keyed State之ListState使用詳細講解(TopN Example)9:58
Flink Keyed State之MapState使用詳細講解7:39
Flink State Time to Live(TTL) 配置及應用15:12
Flink Operator State詳細講解10:14
Flink Stateful Source Collector(CheckpointedFunction)6:41
Flink Stateful Source Operator State Application開發及測試9:44
Flink Broadcast State詳細講解11:29
Flink State Backend之HashMapStateBackend詳細講解8:00
Flink State Backend之EmbeddedRocksDBStateBackend詳細講解11:06
Flink State Questions And Answers3:14

第八部分(Apache Flink 1.17.2 Checkpoints & SavePoints & Exactly Once Semantic)内容概要3:25
Flink Consistent Checkpoints6:14
Flink Consistent Checkpoints算法-Instant Checkpoints7:47
Flink Consistent Checkpoints算法-Periodically Checkpoints5:40
Flink Consistent Checkpoints算法-Chandy-Lamport13:58
Flink Checkpoints and JobManager Checkpoints Coordinator6:19
Flink Checkpoints的基本配置與啟用13:46
Flink Checkpoints Barrier Alignment and Exactly Once配置7:55
Checkpoints Barrier Alignment and At Least Once配置8:12
Flink Checkpoints Barrier Unalignment and Exactly Once配置8:56
Flink savepoints introduction and restoring application state from savepoints8:47
How to apply savepoints when you resume Flink Application12:24
Trigger Savepoints when stop the Flink Job5:50
Flink Checkpoints vs Savepoints9:48
Flink Application End to End Exactly Once Semantics11:00
Flink EOS Application - Idempotent writes6:44
Flink EOS Application - Generic write-ahead-log (WAL) sink16:48
Flink EOS Application - TwoPhaseCommit Sink11:50
Flink work with Apache Kafka implement the End to End Exactly Once Semantics-I12:49
Flink work with Apache Kafka implement the End to End Exactly Once Semantics-II11:56
Flink work with Apache Kafka implement the End to End Exactly Once Semantics-III5:20
Flink New Data Source Collector Specification8:44
Flink New Data Source Collector自定义-I17:54
Flink New Data Source Collector自定义-II15:34
Flink New Data Source Collector自定义-III6:56

Requirements

Apache Hadoop (Mandatory)
Java 1.8+ (Mandatory)
Linux basic commands (Mandatory)
Apache Kafka (Optional)
Apache Kafka Streams Framework(Optional)
Apache Maven(Optional)

Description

非常感謝您能夠購買我的課程，這是一門關於Apache Flink 1.17.x的完全教學視頻，在該視頻中我會非常詳細由淺入深的介紹並且練習每一個Apache Flink框架的使用細節.

Apache Flink 是一個框架和分散式處理引擎，用於對無界和有界資料流進行狀態計算。 Flink 被設計為在所有常見的叢集環境中運行，以記憶體速度和任何規模執行計算。

【課程特色】

代碼驅動
大量的案例
由淺入深
課程內容緊湊
涵蓋絕大多數Apache Flink框架的內容
豐富的綜合案例

[課程大綱]

Apache Flink 1.17.2 快速開始
詳細講解Apache Flink 1.17.2 Job Deployment Mode
詳細講解Apache Flink 1.17.2 運行時架構
詳細講解Apache Flink 1.17.2 DataStream API及即時應用開發
詳細講解Apache Flink 1.17.2 Window API及即時應用開發
深入解讀Apache Flink 1.17.2 Watermark-處理遲到和亂序數據
詳細講解Apache Flink 1.17.2 Stateful operators and Application
詳細講解Apache Flink 1.17.2 Checkpoints & SavePoints & Exactly Once Semantic

希望你能夠喜歡這門課程，經過這門課程的學習，您將成為Apache Flink的專家，並且能夠基於Apache Flink構建複雜即時事件處理應用程式

Who this course is for:

Big Data Engineer
Big Data Developer
Big Data Architecture

Apache Flink 1.17.x DataStream API Streaming即時計算完全教程

What you'll learn

Explore related topics

Course content

Apache Flink 1.17.2 課程介紹3 lectures • 10min

Apache Flink 1.17.2 快速開始14 lectures • 1hr 30min

詳細講解Apache Flink 1.17.2 Job Deployment Mode13 lectures • 1hr 25min

詳細講解Apache Flink 1.17.2 運行時架構14 lectures • 1hr 33min

詳細講解Apache Flink 1.17.2 DataStream API及即時應用開發47 lectures • 6hr 8min

詳細講解Apache Flink 1.17.2 Window API及即時應用開發29 lectures • 4hr 19min

深入解讀Apache Flink 1.17.2 Watermark-處理遲到和亂序數據17 lectures • 2hr 44min

詳細講解Apache Flink 1.17.2 Stateful operators and Application16 lectures • 2hr 14min

詳細講解Apache Flink 1.17.2 Checkpoints & SavePoints & Exactly Once Semantic25 lectures • 4hr 5min

Materials & Source Code Download1 lecture • 1min

Requirements

Description

Who this course is for: