Apache Kafka Series - Confluent Schema Registry & REST Proxy
4.7 (1,334 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
9,312 students enrolled

Apache Kafka Series - Confluent Schema Registry & REST Proxy

Kafka - Master Avro, the Confluent Schema Registry and Kafka REST Proxy. Build Avro Producers/Consumers, Evolve Schemas
Highest Rated
4.7 (1,336 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
9,317 students enrolled
Current price: $83.99 Original price: $129.99 Discount: 35% off
5 hours left at this price!
30-Day Money-Back Guarantee
This course includes
  • 4.5 hours on-demand video
  • 7 articles
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Write simple and complex Avro Schemas
  • Create, Write and Read Avro objects in Java
  • Write a Java Producer and Consumer leveraging Avro data and the Schema Registry
  • Learn about Schema Evolution
  • Perform Schema evolution using the command line and in Java
  • Utilize the REST Proxy using a REST Client
Course content
Expand all 58 lectures 04:29:16
+ Course Introduction
4 lectures 14:53

Short introduction that describes the need of a Schema Registry in Kafka

Preview 04:10

Learn about the course objectives and how the course is structure

Course Structure & Objectives
05:57

Learn about Stephane Maarek, your instructor for this course

About your instructor
02:20

Visit the target architecture with Apache Kafka, the Confluent Schema Registry and the REST Proxy

Preview 02:26
+ Code Download
1 lecture 00:14

Download all the code and tools for the course

Code Download for the Class
00:14
+ Avro Schemas
9 lectures 35:25

Learn what Apache Avro is and how it came to be

Preview 08:19

Learn the Apache Avro primitive types

Avro Primitive Types
02:16

Learn how to define a Record Apache Avro Schema

Avro Record Schema Definition
06:41

Learn what are the Complex types for Apache Avro

Avro Complex Types
05:46

Practice your newly acquired skills by creating a Customer and CustomerAddress Schema

Practice Exercise: Customer & CustomerAddress
04:51

Learn what are the possible Apache Avro logical types

Avro Logical Types
02:46

Learn how to handle decimal numbers in Avro

The complex case of Decimals
04:01

Links to various Avro Schemas you can find on the web

Example of Avro Schemas on the Web
00:08
Avro Schemas Quiz
7 questions

Summary of all the things we have learned so far

Avro Section Summary
00:37
+ Avro in Java
9 lectures 01:02:47

Understand what we're trying to achieve with Avro in Java

Section Objective
01:16

Learn how to create a GenericRecord in Avro

Generic Record in Avro - Hands On
20:10

Learn how to create a SpecificRecord in Avro

Specific Record in Avro - Hands On
13:33

Check point to understand where we are now vs later when using Kafka

Check-in on now vs later in Kafka
01:43

Hands-On practice with the Avro tools

Avro Tools - Hands On
04:15

Learn how to generate an Avro schema using Avro Reflection

Reflection in Avro - Hands On
04:44

Learn the theory behind schema evolution

Schema Evolution - Theory
07:42

Get Hands-On with Schema Evolution in Java

Schema Evolution - Hands On
07:33
Avro in Java Quiz
10 questions

Section summary to understand how we used Java and Apache Avro library to build Avro objects.

Avro Java Section Summary
01:51
+ Setup and Launch Kafka
9 lectures 32:31

Important information regarding the installation process

Important information about Kafka setup
01:03

Install Docker for Mac if you have a recent Mac machine

Docker on Mac (recent versions)
03:54

Install Docker Toolbox on Mac if you have an older machine

Docker Toolbox on Mac (older versions)
05:16

Install Docker on Linux

Docker on Linux (ubuntu as an example)
04:56

Install Docker on recent Windows 10 64bit

Docker on Windows 10 64bit
03:42

Install Docker Toolbox on other Windows version

Docker Toolbox on Windows (older versions)
05:37

Increase the RAM for Docker

Increasing RAM for Docker - PLEASE READ
01:01

Frequently asked and common issues for Docker and Kafka Setup

Frequently Asked Questions - Common Issues
02:26

Start our Kafka Cluster using Docker Compose

Starting Kafka using Docker Compose
04:36
+ Confluent Schema Registry and Kafka
10 lectures 57:37

Introduction to the Confluent Schema Registry

Confluent Schema Registry
07:44

Learn how to use the Kafka Avro Console Producer and Consumer

Kafka Avro Console Producer & Consumer
09:29

Learn how to use Java to write your first Avro Producer

Preview 12:58

Learn how to write an Avro consumer using Java

Writing a Kafka Avro Consumer in Java
07:40

Re-visit what Schema Evolution is before we perform it in Java

Reminder on Schema Evolution
01:04

Writing a V2 Java Kafka Avro Producer with schema evolution!

Writing a V2 Kafka Producer
06:58

Writing a V2 Java Kafka Avro Consumer with schema evolution!

Writing a V2 Kafka Consumer
03:45

Summary for Compatibility Changes

Summary on Compatibility Changes
01:15

Deep Dive to understand how the Schema Registry works behind the scenes

Kafka Schema Registry Deep Dive
04:31

Learn how to efficiently manage schemas by having separate repositories for them

Managing Schemas Efficiently & Section Summary
02:13
Confluent Schema Registry and Kafka Quiz
7 questions
+ Confluent REST Proxy
10 lectures 39:27

Introduction to understand what is the REST Proxy and what is its purpose

Kafka REST Proxy Introduction and Purpose
03:54

Understand the difference of the v1 vs the v2 API for the Confluent REST Proxy

V1 vs V2 APIs
02:18

Setup Insomnia on your Machine to have a REST Client

Insomnia Setup (REST Client)
01:26

Understand how to list topics or view a specific topic's detail using the REST Proxy

Topic Operations
03:26

Produce Binary data that's base64 encoded using the REST Proxy

Producing in Binary with the Kafka REST Proxy
07:15

Consume Binary data that's base64 decoded using the REST Proxy

Consuming in Binary with the Kafka REST Proxy
05:07

Learn how to use the Confluent REST Proxy to produce data in JSON

Producing in JSON with the Kafka REST Proxy
03:18

Learn how to use the Confluent REST Proxy to consume data in JSON

Consuming in JSON with the Kafka REST Proxy
03:25

Learn how to use the Confluent REST Proxy to produce data in Avro

Producing in Avro with the Kafka REST Proxy
05:36

Learn how to use the Confluent REST Proxy to consume data in Avro

Consuming in Avro with the Kafka REST Proxy
03:42
Confluent REST Proxy Quiz
7 questions
+ Annexes
3 lectures 22:52
What are Annexes?
00:13

End to End example for Avro, using Producer, Kafka Connect and Kafka Streams

Full Avro End to End: Kafka Producer + Kafka Connect + Kafka Streams
19:00

Learn how to find documentation for the REST Proxy and how to Scale it in Production

Kafka REST Proxy Installation and Scaling - Overview
03:39
+ Next Steps
3 lectures 03:28

Pointers for the next steps in your Kafka learning

What's next?
01:43
THANK YOU!
01:32

Special coupons for my other Kafka Courses

Bonus Lecture: Special Coupons to my Other Courses
00:13
Requirements
  • Good knowledge of Apache Kafka
  • Some Java knowledge
  • A recent computer (Windows / Linux / Mac OS X) with 4GB of RAM
  • Understand JSON data format
  • Knowledge of REST APIs is better
Description

The new volume in the Apache Kafka Series!

Learn Apache Avro, the Confluent Schema Registry for Apache Kafka and the Confluent REST Proxy for Apache Kafka. Join hundreds of knowledge savvy students into learning some of the most important components in a typical Apache Kafka stack.

Take a look at these student reviews...

★★★★★  "Excellent course. A must for Apache Kafka users." – Mayank

★★★★★  "Beautiful explanation of Avro and how to use it with Apache Kafka. This knowledge enhancing like all your other courses. We have Apache Kafka from Confluent in our enterprise and this is what I was exactly looking for. Thanks a lot." – Vinod

★★★★★   "Stephane Maarek is truly remarkable instructor! His instructional videos do not disappoint, they keep me wanting more and provide exactly the Kafka skills needed to be successful." – Troy

----------------------------

Apache Avro is one of the most powerful and most popular fast data serialisation mechanism with Apache Kafka

> Schema is a first class citizen of the data
> Data serialisation format that is lightweight
> Documentation is embedded in the schema
> Easy to read and write using many languages
> Fast & Big data de-facto data serialisation format


Confluent Schema Registry for Apache Kafka is the de-facto standard way of storing Avro Schemas for your Apache Kafka Topics.

> Stores a versioned history of all your schemas in Apache Kafka
> Supports and enforces conditions for schema evolution (backward, forward, full compatibility)
> Kafka Avro Serialiser and Deserialiser automatically integrate with Apache Kafka and Confluent Schema Registry
> Make your messages much smaller and lighter! Perfect for increasing Apache Kafka throughput and performance

Confluent REST Proxy is the perfect way to communicate for sending Avro data using non Java languages to Apache Kafka. 

> Write and Read binary, JSON and Avro data to Apache Kafka using an HTTP REST API
> Interact with Apache Kafka using any programming language (not just Java)
> Consult topic list and topic metadata in Apache Kafka

Instructor

Stephane Maarek is the instructor of this course. He is an Apache Kafka Expert, and the author of the highly-rated Apache Kafka Series on Udemy, having taught already to 40000+ students and received 12000+ reviews.   He shares all his Apache Kafka knowledge on the platform, taking the time to explain every concept and provide students with both theoretical and practical dimensions. You are in good hands!

Hands On & Theory Based Course

Note: This course is comprehensive, and the Apache Kafka related bits do not start before the Kafka Schema Registry section. This is to ensure Avro Schema and Avro in Java is fully understood before moving to the Confluent Schema Registry for Apache Kafka. 

This course is the first and only available Kafka Schema Registry and Kafka REST Proxy course on the web. Get it now to become an Apache Kafka expert!

Section outline:

  1. Course Introduction: Let's learn why we need Schema, and what the target Kafka architecture will look like. 

  2. Avro Schemas: Master how to write simple and complex Avro Schemas for your Kafka messages, as well as Schema Evolution rules and guidelines. 

  3. Avro in Java: Learn how to create Avro records using GenericRecord, SpecificRecord, and ReflectedRecord. Also perform your first schema evolution in Java. 

  4. Setup and Launch Kafka: Install Docker and use Docker Compose to start your Apache Kafka Cluster that will contain the Confluent Schema Registry and the Kafka REST Proxy.

  5. Confluent Schema Registry and Kafka: Learn what is the Confluent Schema Registry, how it works. Learn to use the Kafka Avro Console Producer & Consumer, and write your first Apache Kafka Avro Java Producer and Avro Java Consumer. Perform a fully compatible schema evolution

  6. Confluent REST Proxy: Learn how to use the REST Proxy with a REST Client (Insomnia) in order to interface with Apache Kafka using REST. Understand how to read Kafka topic metadata, produce and consume binary, JSON and Avro data

  7. Annexes: Extra Apache Kafka lectures to push your learning further

This Course Also Comes With:

✔ Lifetime Access to All Apache Kafka course Future Updates

✔ A responsive instructor in the Q&A Section

✔ Links to interesting articles, and lots of good code to base your next applications onto

✔ Udemy Certificate of Completion Ready for Download

✔ A 30 Day "No Questions Asked" Money Back Guarantee!

Learn The Confluent Components Now!

Apache Kafka is increasingly becoming a must-have skill, and this course will set you up for fast success using Avro in Kafka, and the Confluent Components - the Kafka Schema Registry and the Kafka REST Proxy. The job market will need people with your newly acquired skillset!

"Take This Course" risk free and learn the Confluent Schema Registry and Kafka REST Proxy now!

Who this course is for:
  • Developers who would like to learn how to write and run an application that leverages Avro and the Schema Registry
  • Architects who want to understand the role of the Schema Registry and REST Proxy in an enterprise pipeline
  • This course is intended for people who fully understand the basics of Kafka
  • People who want to learn the Confluent Components: Schema Registry and REST Proxy