
Learn Debezium, an open source change data capture platform, that streams row-level database changes in order to Kafka, enabling real-time data sync across MySQL, PostgreSQL, and more.
Capture database changes in real time with change data capture, replacing batch snapshots, and feed them into a destination via a Debezium-driven Kafka Connect pipeline.
Explore the Debezium for MySQL division connector and its topologies. Learn how binlog monitoring and position tracking enable reliable changes across standalone, primary and replica, and high availability clusters.
Install kafka connect in distributed mode, add the mysql connector plugin, and verify availability via curl and jq on port 8083.
Install MySQL 5.7 on Docker by pulling the image and running a container with port 3306. Retrieve the root password, log in, and grant full root privileges.
Create the Debezium MySQL connector via a POST to the Kafka Connect REST API, configuring a unique server name, single task, database include list, and Avro with schema registry.
Learn how to use Kafka Connect APIs to create, delete, and manage connectors, and fetch connector details, status, and configuration through curl requests.
Explore the schema registry APIs to list schemas, view supported schema types (JSON, protocol, buffer, AVRO), and retrieve subject versions and schemas by id or latest version.
Learn how to send signals to division in Debezium, including log and snapshot signals, configure a signal table and data collection, and understand incremental snapshot behavior.
Explore Debezium in production architecture by comparing standalone versus cluster mode, illustrating fault-tolerant Kafka Connect deployments and automatic task recovery across multiple servers.
Learn to install Debezium in cluster mode by configuring a two-server MySQL and Kafka Connect distributed setup with Zookeeper, Schema Registry, and proper properties.
Demonstrate change data capture with Debezium in cluster mode, showing fault-tolerant, distributed kafka connect connectors across cluster hosts. Create ecommerce, inventory, and complaints databases with key tables, illustrating balanced deployment.
Debezium is an open source distributed platform for change data capture. It is a well-known tool for capturing the data change in your databases, and bring it to your apps as soon as they take place.
Debezium is used in production by a wide range of companies and organizations. Debezium has the capability to capture data change across different databases like MySQL, PostgresSQL, SQL Server, Oracle DB, MongoDB, Cassandra, DB2 and Vitess.
In this course, we will learn in detail how to use Debezium with MySQL database.
What's included in the course?
Complete course on Debezium capabilities explained from Scratch.
Explored in detail how you can integrate Debezium with MySQL server.
Each and Every Debezium concept is explained with a HANDS-ON.
Detailed steps on Debezium setup in both standalone and production scenarios.
Includes even those concepts, the explanation to which is not very clear even in Debezium official documentation.
Related Commands and Datasets used in lectures are attached in the course for your convenience.
Also contains an additional module on how to use Debezium in production.
Who should take this course?
Data engineers who are specializing in Data Ingestion.
Computer graduate interested in mastering Data Engineering.
Data architects who want to understand in detail the concept of change data capture.