Confluent ksqlDB complete course-Build streaming app by SQL

Explore theoretical knowledge of covalent, physical and keizoku, and the features of key circle and key tokudb. Examine ksqldb and the Kafka stream framework, internal components, deployment models—interactive and headless.

Explore how ksqlDB and the Kafka Streams framework cooperate to build streaming apps on Kafka topics, comparing SQL-driven processing with low-level stream processing for flexible stateful operations.

Explore ksqlDB architecture and internal components, including the stream processing engine, materialized views, and the REST API for building streaming apps by defining statements with connectors and Kafka topics.

Learn how to deploy a ksqlDB cluster in interactive or headless modes, connect via REST and CLI, and manage persistent queries with command topics and configuration topics.

Learn to set up the ksqlDB development environment with Zookeeper, Kafka, and ksqlDB server clusters, run your first streaming app, and grasp the ksqlDB query lifecycle.

Install and configure zookeeper and kafka cluster for a single-instance setup, start zookeeper and kafka, then create a test topic and perform producer-consumer testing.

Install and configure the Confluent ksqlDB server in a local environment, set service and listener properties, start the server, verify with a health check, and prep for the streaming app.

Define variables and configurations, create a source stream, build the topology, and start the Kafka streams using ksql statements, with hands-on ddl and query demos.

Learn to batch submit queries in ksqlDB using the run script command, create streams and tables, and run predefined queries from a script file.

Demonstrate ksqlDB headless deploy mode, including setting properties, creating streams, deriving the user stream, and verifying results by consuming the transformed user topic.

Understand the ksqlDB query statement execution lifecycle from creating streams or tables with sql to building logic and physical plans, then running and managing the streaming application.

Explore stream processing basics with ksqlDB, from data types and schema registry to source collections and derived queries, and build a word count app using persistent queries and Kafka topics.

Learn ksqlDB simple and compound data types, modeling a person stream with name, age, and address as a struct and friends as a map or array.

Learn how ksqlDB supports custom data types using the create tabs statement to define address as a structure and reuse type definitions while managing streams.

Explore ksqlDB's delimited serialization format for csv data by defining key and value delimiters, including the default comma, creating a topic and stream, and querying records from a csv-based stream.

Compare ksqlDB json and adjacent sr formats, and learn to install and configure confluent schema registry, connect it to Kafka, set the _schemas topic, and verify integration.

Integrate confluent schema registry with ksqlDB and use the json schema registry serialization format to produce employee records via a Java producer and create a stream with json schema.

Explore ksqlDB collections by comparing streams and tables, where streams are immutable event sequences and tables store the latest state with tombstones, lookup tables, and derived collections.

Learn ksqlDB source streams and tables syntax through practical examples, including create, replace, not exists handling, and querying fields and metadata. Discuss read-only source behavior and a pull query discrepancy.

Create ksqlDB source collections for streams and tables, using parameter keys and primary keys, note permanent keyword differences, and build tables via select all or post queries with multiple keys.

Define ksqlDB streams and tables with clause to set a custom timestamp column and its format, and understand event time, injected time, and process time for windowing and joins.

Learn how to control ksqlDB time semantics by using a custom timestamp column, specifying time step and timestamp format, and deriving streams to override defaultKafka record timestamps.

Explore ksqlDB basics by filtering and transforming data in the person stream with transient queries, selects, where conditions, between, limit, and export to individual records.

Master coalesce, if, not, and case expressions to handle nulls in collection where clause queries, then apply these techniques to build streaming apps with SQL.

Create derived collections by deriving streams and tables from existing ones using CSAS and CTAS syntax, including source collections, partition by, and persistent queries that recover after restarts.

Explore persistent queries in ksqlDB by writing results back to a Kafka topic, explicitly naming topics, configuring partitions and formats, and creating streams and tables based on derived collections.

Explore how the explain statement reveals the execution plan and streams topology for a sql expression, showing source text line streams, flat map transforms, and sink to the word topic.

Define variables and substitution in ksqlDB statements, including proper single-quote quoting and value substitution. Use these variables to create streams with specified columns, partitions, and formats.

Explore ksqlDB collection joins and windowed processing for streams and tables. Learn inner, left, and full joins and how windowing applies to unbounded streams.

Demonstrates join operations in ksqlDB by joining two streams with a one-minute window: inner, left, and full joins, using stream schemas, timestamps, and on conditions to produce joined results.

Inner and left joins between a stream and a table are supported by ksqlDB, using lookup semantics; outer joins aren’t available, and downstream outputs react only to stream updates.

Demonstrates ksqlDB table primary key joins, including inner, left, and full joins, and contrasts them with foreign key joins, while noting windowed joins and eventual consistency.

Explore how ksqlDB uses the group by clause and aggregate functions to compute summaries over streams and tables, storing results in mutable structures for live updates.

Explore the group by clause with having, counting people by title and extracting the top per title, while noting aggregate limitations and stream versus table behavior in Kafka streams.

Explore how ksqlDB and Kafka Streams handle windowed aggregations, emitting final results only when a window closes, using the suppress operation to withhold updates to downstream systems.

Configure hopping time windows in ksqlDB by setting a one-minute window size and a 30-second hop interval to analyze real-time hot words, such as Twitter keywords, with overlapping windows.

Configure and start a Kafka Connect distributed cluster, set bootstrap servers and group ID, enable plugins, verify with REST health checks, and prepare to register connectors for ingesting filesystem data.

Install and configure file source plugins for Kafka Connect, restart the worker, and create a FileSource connector to stream files into a topic with schema metadata visible.

Install and configure the JDBC source connector for Kafka Connect, create two connectors to ingest patient and room table data into Kafka topics for real-time health monitoring and alerting.

Learn how to develop ksqlDB user defined functions, including scalar UDFs, UDTFs, and UDAFs, and apply lambda functions in SQL.

Learn to implement a bmi udf as a scalar function in ksqldb by creating a java maven project, annotating the function, and deploying it as an extension for the server.

Define a custom ksqlDB udtf (tabular function) that outputs multiple rows from a single input by using split and explode on a delimiter.

Interact with the local ksqlDB using the rest api to list streams and tables, create or drop resources, run push and pull queries, insert data, and monitor health and properties.

Explore a lightweight Java client for ksqlDB that sends requests from your Java app, supports pull and push queries, and manages streams, tables, and persistent queries.

Demonstrates using the ksqlDB java client to list streams and print stream and topic details, then executes a push query with a subscriber for asynchronous results.

Learn how to use the ksqlDB Java client to execute synchronous stream queries and batch pull queries, print results to the console, and manage the client lifecycle.