Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Development Tools No-Code Development
Business
Entrepreneurship Communications Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certification Network & Security Hardware Operating Systems Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design Design Thinking 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition Yoga Mental Health Dieting Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Teacher Training Test Prep Other Teaching & Academics
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Cisco CCNA Amazon AWS AWS Certified Developer - Associate CompTIA Security+
Photoshop Graphic Design Adobe Illustrator Drawing Digital Painting InDesign Character Design Canva Figure Drawing
Life Coach Training Neuro-Linguistic Programming Mindfulness Personal Development Personal Transformation Meditation Life Purpose Coaching Neuroscience
Web Development JavaScript React CSS Angular PHP Node.Js WordPress Python
Google Flutter Android Development iOS Development Swift React Native Dart Programming Language Mobile Development Kotlin SwiftUI
Digital Marketing Google Ads (Adwords) Social Media Marketing Google Ads (AdWords) Certification Marketing Strategy Internet Marketing YouTube Marketing Email Marketing Retargeting
SQL Microsoft Power BI Tableau Business Analysis Business Intelligence MySQL Data Analysis Data Modeling Big Data
Business Fundamentals Entrepreneurship Fundamentals Business Strategy Online Business Business Plan Startup Freelancing Blogging Home Business
Unity Game Development Fundamentals Unreal Engine C# 3D Game Development C++ 2D Game Development Unreal Engine Blueprints Blender
30-Day Money-Back Guarantee

This course includes:

  • 4.5 hours on-demand video
  • 5 articles
  • Full lifetime access
  • Access on mobile and TV
Development Software Engineering Apache Kafka

Apache Kafka Series - Kafka Connect Hands-on Learning

Kafka Connect - Learn How to Source Twitter Data, Store in Apache Kafka Topics & Sink in ElasticSearch and PostgreSQL
Rating: 4.5 out of 54.5 (2,770 ratings)
22,102 students
Created by Stephane Maarek | AWS Certified Solutions Architect & Developer Associate
Last updated 1/2021
English
English [Auto], Italian [Auto]
30-Day Money-Back Guarantee

What you'll learn

  • Configure and run Apache Kafka Source and Sink Connectors
  • Learn concepts behind Kafka Connect & the Kafka Connect architecture
  • Launch a Kafka Connect Cluster using Docker Compose
  • Deploy Kafka Connectors in Standalone and Distributed Mode
  • Write your own Kafka Connector
Curated for the Udemy for Business collection

Course content

9 sections • 55 lectures • 4h 26m total length

  • Preview01:09
  • Preview01:20
  • Preview01:55
  • Preview02:20

  • Code Download
    00:14

  • What is Kafka Connect?
    Preview04:50
  • Preview03:20
  • Preview03:03
  • Standalone vs Distributed Mode
    Preview02:11
  • Preview02:58

  • Important information about installation
    01:01
  • Docker on Mac (recent versions)
    03:54
  • Docker Toolbox on Mac (older versions)
    05:16
  • Docker on Linux (Ubuntu as an example)
    04:56
  • Docker on Windows 10 64bit
    03:42
  • Docker Toolbox on Windows (older versions)
    05:37
  • Important note for Docker Toolbox users
    00:28
  • Starting Kafka Connect Cluster using Docker Compose
    03:40

  • It's not working! What to do?
    02:49
  • Where to view logs?
    04:05
  • Asking a question in the Q&A
    02:39

  • Kafka Connect Source Architecture Design
    01:14
  • FileStream Source Connector - Standalone Mode - Part 1
    09:05
  • FileStream Source Connector - Standalone Mode - Part 2
    07:56
  • FileStream Source Connector - Distributed Mode
    11:29
  • List of Available Connectors
    05:04
  • Twitter Source Connector - Distributed Mode - Part 1
    07:37
  • Twitter Source Connector - Distributed Mode - Part 2
    06:11
  • Section Summary
    01:36

  • Kafka Connect Sink Architecture Design
    01:43
  • ElasticSearch Sink Connector - Distributed Mode - Part 1
    08:58
  • ElasticSearch Sink Connector - Distributed Mode - Part 2
    07:31
  • Kafka Connect REST API
    11:08
  • JDBC Sink Connector - Distributed Mode
    13:06

  • Goal of the section: GitHubSourceConnector
    01:34
  • Finding the code and installing required software
    03:09
  • Description of the GitHub Issues API
    03:55
  • Using the Maven Archetype to get started
    05:43
  • Config Definitions
    12:31
  • Connector Class
    06:57
  • Writing a schema
    07:31
  • Data Model for our Objects
    06:10
  • Writing our GitHub API HTTP Client
    07:03
  • Source Partition & Source Offsets
    04:22
  • Source Task
    13:40
  • Building and running a Connector in Standalone Mode
    04:59
  • Command for next lecture
    00:20
  • Deploying our Connector on the Landoop cluster
    06:30
  • More Resources for Developers
    01:57

  • Setting up Kafka Connect in Production (1/2)
    14:19
  • Setting up Kafka Connect in Production (2/2)
    06:15
  • What's next?
    01:46
  • Congratulations and Thank you!
    01:18
  • THANK YOU!
    01:32
  • Bonus Lecture: Student Special Coupons for my Other courses
    01:01

Requirements

  • Good understanding of Kafka Basic is REQUIRED to take this course
  • A recent Windows / Mac / Linux machine with minimum 4GB of RAM, 5 GB of disk space
  • Good to have knowledge about Linux command line
  • Good to have knowledge about Docker
  • A note book and pen is recommended to take notes during the entire course

Description

A Comprehensive and Brand New Course for Learning Apache Kafka Connect Framework with Hands-on Training – (Launched in April 2017)

Kafka Connect is a tool for scalable and reliable streaming data between Apache Kafka and other data systems. Apache Kafka Connect is a common framework for Apache Kafka producers and consumers.

Apache Kafka Connect offers an API, Runtime, and REST Service to enable developers to define connectors that move large data sets into and out of Apache Kafka in real time. It inherits strong concepts such as fault-tolerance and elasticity thanks to being an extension of Apache Kafka. Kafka Connect can ingest entire databases, collect metrics, gather logs from all your application servers into Apache Kafka topics, making the data available for stream processing with low latency.

Kafka Connect standardises integration of other data systems with Apache Kafka, simplifying connector development, deployment, and management.

In this course, we are going to learn the Kafka Connector deployment, configuration and management with hands-on exercises. We are also going to see the distributed and standalone modes to scale up to a large, centrally managed service supporting an entire organisation or scale down to development, testing, and small production deployments. The REST interface to submit and manage connectors to your Kafka Connect cluster via an easy to use REST API’s.

Students Loved this course – Students have rated the course with 5 stars...!!

Tin Pavlinic says “This course is an excellent resource for those getting started with Apache Kafka-Connect. Stephane explains concepts very clearly and provides plenty of practical examples. The setup is easy. You can follow the examples along on your own machine, and then experiment further. I highly recommend this Apache Kafka course.” ★★★★★

Alan V says "This course is GREAT: 1. The teacher provides clear explanations of the role to Kafka Connector in Kafka Streaming Solution architecture. 2. The course uses docker-compose.yml to help learners quick build a practice environment. 3. The course is well organized and easy to understand the idea, behind Kafka Connector. The teacher guides you how to use the tool step by step. 4. The provided shell commands are all well commented and steps by steps to guide users. If you read the comments in the shell script files after the course, it will remind you what the teacher just taught in the videos. 5. The teacher is very responsible to learners' questions and very helpful. If you are a professional developer, I highly recommend you to attend this course." ★★★★★

Sagar Joshi says "It was very engaging and interesting course. I recommend taking this Apache Kafka course. Stephane has provided all the required details and its really a hands-on training course. Thank you..!!" ★★★★★

Why I should take this course?

  • My aim is to make you confident to work with Apache Kafka Connector as soon as possible by providing a hands-on training, source data from Twitter and sinking it directly into ElasticSearch and PostgreSQL database
  • In this course, you will learn what is Kafka Connect, Kafka Connect architecture, how to deploy an Apache Kafka Connector in standalone and in distributed mode, how to setup and launch Kafka Connect Cluster using Docker compose
  • You will be able to Deploy Kafka Connect Source Connectors and Kafka Connect Sink Connectors
  • You have strong support from the instructor, I am very happy to answer all your questions and queries inside the course Q&A sections
  • You have life-time access to this course and a 30-day satisfaction guaranteed with this course

Overview of the Course Contents -

Section 1 – Course Introduction: In this section, we are going to see the required pre-requisites for this course and for Apache Kafka Connect. We are also going to talk about the objectives and the structure of the course.

Section 2 - Apache Kafka Connect Concepts: In this section, we will learn about what is Kafka Connect, Apache Kafka Connect architecture, we will talk about Connectors, Configuration, Tasks, Workers. We are also going to learn the difference between the standalone vs distributed mode of the Kafka Connect.

Section 3 – Setup and Launch Kafka Connect Cluster: In this sections, we will learn how to install Docker on your machine and get started with Apache Kafka Connect in the simplest way possible using Docker Compose.

Section 4 – Apache Kafka Connect Data Source – Hands-on: In this section, we will gain some practical experience on Kafka Connect Data Source. We will learn the Kafka Connect Data Source architecture and the list of available connectors. We will have some hands-on practice and learning on File Stream Source Connector both standalone and distributed, and on Twitter Source Connectors…!!!

Section 5 - Apache Kafka Connect Data Sink – Hands-on: In this section, we will gain some practical experience on Kafka Connect Data Sink. We will learn the Kafka Connect Data Sink architecture, Apache Kafka Connect REST API’s and we will have some hands-on practice and learning on Elastic Search Sink Connector and on JDBC Sink Connectors…!!!

Section 6 – Next Steps: In this section, we are going to conclude the course and going to see what is next step you can follow.

You have lifetime access to the course and 30 days’ money back guarantee, so what are you thinking? Click on “Enroll Now” button now and see you inside the course…!!!

Who this course is for:

  • Developers who wants to learn Apache Kafka Connect Framework and get hands-on with it
  • Professionals who are good at Apache Kafka ecosystem and basic core concepts
  • Architects who want to understand how Kafka Connect fits in their Solution Architecture

Featured review

M. Hamza Nasir
M. Hamza Nasir
239 courses
11 reviews
Rating: 5.0 out of 5a year ago
I took Kafka Beginners and Kafka Streams course of Stephane Maarek. Now, I am taking the Kafka Connectors course. Stephane is the go to guy if you REALLY need to learn Kafka without any headache. Loving his courses!

Instructor

Stephane Maarek | AWS Certified Solutions Architect & Developer Associate
Best Selling Instructor, Kafka Guru, 9x AWS Certified
Stephane Maarek | AWS Certified Solutions Architect & Developer Associate
  • 4.7 Instructor Rating
  • 205,855 Reviews
  • 680,852 Students
  • 38 Courses

Stephane is a solutions architect, consultant and software developer that has a particular interest in all things related to Big Data, Cloud & API. He's also a many-times best seller instructor on Udemy for his courses in Apache Kafka and AWS.

[See FAQ below to see in which order you can take my courses]

Stéphane is recognized as an AWS Hero and is an AWS Certified Solutions Architect Professional & AWS Certified DevOps Professional. He loves to teach people how to use the AWS properly, to get them ready for their AWS certifications, and most importantly for the real world.

He also loves Apache Kafka. He sits on the 2019 Program Committee organizing the Kafka Summit in New York, London and San Francisco. He is also an active member of the Apache Kafka community, authoring blogs on Medium and a guest blog for Confluent.  

During his spare time he enjoys cooking, practicing yoga, surfing, watching TV shows, and traveling to awesome destinations!

FAQ: In which order should you learn?...

AWS Cloud: Start with AWS Certified Solutions Architect Associate, then move on to AWS Certified Developer Associate and then AWS Certified SysOps Administrator. Afterwards you can either do AWS Certified Solutions Architect Professional or AWS Certified DevOps Professional, or a specialty certification of your choosing. Once ready, you can learn AWS Lambda and AWS CloudFormation in depth, or do the AWS Big Data certification.

Apache Kafka: Start with Apache Kafka for Beginners, then you can learn Connect, Streams and Schema Registry if you're a developer, and Setup and Monitoring courses if you're an admin. Both tracks are needed to pass the Confluent Kafka certification.

gRPC: First do the protocol buffers course, then move on to gRPC Java or gRPC Golang course.


In order to share knowledge I am not able to respond to private messages on Udemy. If you're a student, please ask questions inside the course. Thanks for understanding!

  • Udemy for Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Featured courses
Udemy
© 2021 Udemy, Inc.