Apache Flink Relational Programming using Table API and SQL
What you'll learn
- Apache Flink Table API
- Apache Flink SQL Interface
- Apache Flink with Python (PyFlink)
- Batch Data Processing
- Stream Data Processing
Requirements
- Previous experience with Python programming
- Basic Understanding of Operating Systems and Docker
- Basic Understanding of Distributed Computing
Description
Apache Flink is widely growing in popularity for its ability to perform advanced stateful computations in a way that scales to meet the demands of both high throughput and high performance use cases. Not only is Apache Flink very scalable and performant it also integrates with a wide variety of source and sink data systems like flat files (CSV,TXT,TSV), Databases, and Message Queues (Kafka, AWS Kinesis, GCP Pub/Sub, RabbitMQ).
In this course students will learn to harness the power of Apache Flink which is a modern distributed computing framework providing a unified approach to both batch and streaming data processing workloads. This course specifically focuses on the relational programming paradigm exposed through Apache Flink's Table API and SQL interface (with examples in Python) offering intuitive yet powerful abstractions to process vast amounts of data in either bounded (batch) or unbounded (streaming) sources.
Students learn batch processing with Flink through many examples of consuming, processing, and producing results from/to the filesystem in CSV format.
Students also learn stream processing with Flink through several examples consuming, processing and producing results from/to Apache Kafka running in a local Dockerized Kafka cluster.
Apache Flink offers support for developing Flink applications with the Table API and SQL interface in Java, Scala and Python. However, this course focuses on using the Python bindings for Apache Flink. The focus on Python for this course was chosen due to the popularity of the Python programming language, particularly in the big data engineering ecosystem, but also due to the underrepresentation of Python in existing Apache Flink courses which primarily cover the Java and Scala APIs of Flink.
Who this course is for:
- Data centric Python developers
Instructor
- 4.5 Instructor Rating
- 978 Reviews
- 9,852 Students
- 7 Courses
Experienced Software Engineer with a demonstrated history of working in High-Tech Enterprises like Digital Media, Biotech, and Financial Services. Skilled at crafting well engineered solutions (responsive, scalable, fault tolerant with sensible observability) across multiple technology frameworks and languages spanning all layers of the enterprise. Server-side development experience includes Python, NodeJS/Typescript, and Java paired with client technologies like JavaFX, ElectronJS, VueJS and ReactJS. Early career exposure to the challenges of high volume high complexity data in the sciences provided a strong foundation in analytics and data engineering spanning simple automation to distributed computing technologies like Celery, Redis, Spark, Hive, Kafka, Kinesis and, deep understanding of relational databases (PostgreSQL especially).