Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Development Tools No-Code Development
Business
Entrepreneurship Communications Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certification Network & Security Hardware Operating Systems Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design Design Thinking 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition Yoga Mental Health Dieting Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Teacher Training Test Prep Other Teaching & Academics
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Cisco CCNA Amazon AWS CompTIA Security+ AWS Certified Developer - Associate
Graphic Design Photoshop Adobe Illustrator Drawing Digital Painting InDesign Character Design Canva Figure Drawing
Life Coach Training Neuro-Linguistic Programming Mindfulness Personal Development Meditation Personal Transformation Life Purpose Coaching Neuroscience
Web Development JavaScript React CSS Angular PHP WordPress Node.Js Python
Google Flutter Android Development iOS Development Swift React Native Dart Programming Language Mobile Development Kotlin SwiftUI
Digital Marketing Google Ads (Adwords) Social Media Marketing Google Ads (AdWords) Certification Marketing Strategy Internet Marketing YouTube Marketing Email Marketing Retargeting
SQL Microsoft Power BI Tableau Business Analysis Business Intelligence MySQL Data Analysis Data Modeling Big Data
Business Fundamentals Entrepreneurship Fundamentals Business Strategy Online Business Business Plan Startup Freelancing Blogging Home Business
Unity Game Development Fundamentals Unreal Engine C# 3D Game Development C++ 2D Game Development Unreal Engine Blueprints Blender
30-Day Money-Back Guarantee

This course includes:

  • 15 hours on-demand video
  • Full lifetime access
  • Access on mobile and TV
IT & Software Other IT & Software Apache Kafka

Build streaming applications using Apache Kafka and Scala

Learn Apache Kafka using Scala as programming language.
Rating: 3.8 out of 53.8 (96 ratings)
806 students
Created by Durga Viswanatha Raju Gadiraju, Itversity Support, Kamlesh Patidar
Last updated 12/2018
English
English [Auto]
30-Day Money-Back Guarantee

What you'll learn

  • Setup Kafka locally
  • Understand multi node Kafka cluster setup
  • Develop Scala based applications which can produce messages into Kafka topic
  • Develop Scala based applications which can consume messages from Kafka topic
  • Integration of Flume and Kafka
  • Overview of Kafka Connect
  • Overview of Kafka Streams and KSQL

Requirements

  • Programming Skills using Scala as Programming Language
  • Basic Linux Skills
  • 64 bit Computer with at least 4 GB RAM, dual core and enough storage
  • 64 bit Operating System - Windows 10 or Mac or Linux

Description

In this comprehensive course, you will be learning

  • Setting up of development environment using IntelliJ for Kafka based applications

  • Setup Kafka locally along with Zookeeper and validate

  • Step by step instructions to setup multi broker Kafka setup

  • In depth details about Kafka core concepts

  • Use Kafka Producer API with Scala to produce messages to Kafka topic from web application

  • 3rd party plugins such as Kafka connect and Flume to get data from web server logs into Kafka topic

  • Use Kafka Consumer API with Scala to consume messages from Kafka topic

  • 3rd party plugins such as Kafka connect and Spark streaming to consume messages from Kafka topic

Who this course is for:

  • Application Developers especially Scala Developers who want to learn Kafka to build streaming pipelines
  • Data Engineers who want to build streaming pipelines

Course content

10 sections • 94 lectures • 15h 3m total length

  • Preview07:42
  • Preview05:20
  • Setup IntelliJ with Scala Plugin
    04:15
  • Develop Hello World Program using IntelliJ with Scala Plugin
    06:52
  • Setup SBT to build jar files for Scala applications
    05:45
  • Setup Ubuntu using Windows Subsystem for Linux
    07:31
  • Setup Development Environment for Scala - Conclusion
    01:20

  • Setup Data Sets
    07:45
  • Setup Kafka locally on Windows or Mac or Linux
    14:38
  • Validate Kafka setup by creating test topic
    10:26
  • Validate Kafka setup using simulated web server logs (gen_logs)
    09:18
  • Apache Kafka - Multinode cluster - Overview
    19:20
  • Apache Kafka - Multinode cluster - Review Properties
    10:03
  • Apache Kafka - Concepts
    14:36
  • Scenarios - Single Partition Topic - One Consumer
    10:14
  • Scenarios - Single Partition Topic - Multiple Consumers
    06:10
  • Scenarios - Multi Partition Topic - Single Consumer
    07:12
  • Scenarios - Multi Partition Topic - Multiple Consumers
    08:30
  • Messages and Message Format
    08:56

  • Introduction
    04:40
  • Pre-requisites for multi cluster setup
    12:39
  • Setup Kafka and Zookeeper binaries
    10:03
  • Configure Zookeeper
    08:54
  • Zookeeper Commands
    11:10
  • Configure Kafka Brokers
    07:36
  • Validate Kafka
    14:44

  • Introduction
    01:02
  • Revision of Kafka Topic
    02:36
  • Create Project - KafkaWorkshop
    04:34
  • Add Dependencies for Kafka APIs
    05:46
  • Externalize Properties using typesafe config
    10:51
  • Producer APIs using REPL
    22:56
  • Producer APIs using Program
    13:45
  • Consumer APIs using REPL
    13:38
  • Consumer APIs using Program
    10:29

  • Introduction
    09:34
  • Characteristics of Kafka Topic
    10:21
  • Producer APIs - ProducerRecord Functions
    10:04
  • Producer APIs - Produce Log Messages into Partitioned Topic - with out key
    13:42
  • Producer APIs - Produce Log Messages into Partitioned Topic - using Key
    16:40
  • Setup maxmind geoip database and API dependencies
    14:25
  • Producer APIs - Produce Log Messages into Partitioned Topic - using Partition
    13:27
  • Producer APIs - More important properties from ProducerConfig
    12:43
  • Producer APIs - Build as Application
    13:23
  • Consumer APIs - Review of ConsumerConfig, KafkaConsumer and ConsumerRecord
    17:25
  • Consumer APIs - Subscribing to a Topic
    19:56

  • Introduction
    03:50
  • Flume Overview
    15:35
  • Setup gen_logs
    09:53
  • Develop first Flume Agent
    18:09
  • Understand Source, Sink and Channel
    20:44
  • Simple Multi Agent Flow
    17:38
  • Consolidation
    03:42
  • Multiplexing and Replicating
    07:24
  • Getting Log Data into HDFS - Define Source
    14:24
  • Getting Log Data into HDFS - Define Sink
    18:04
  • Getting Log Data into HDFS - Understand Escape Sequences
    07:01
  • Getting Logs into Kafka - Define Kafka Sink
    13:15
  • Getting Logs into Kafka - Using File Channel
    13:26
  • Limitations and Conclusion
    07:38

  • Overview Of Spark Streaming (Legacy)
    05:40
  • Overview of DStreams and APIs
    01:32
  • Typical Development Life Cycle for Spark Streaming Applications
    01:25
  • Develop Streaming Department Count using socketTextStream
    16:53
  • Develop Streaming Department Count using Kafka
    12:18
  • Spark Streaming - Windowing Functions Overview
    04:16

  • Setup HBase Locally
    01:48
  • Basic NoSQL Concepts and HBase
    10:23
  • CRUD Operations in HBase using HBase Shell
    13:18
  • Setup sbt Project for Scala and HBase
    03:57
  • CRUD Operations using Scala REPL
    13:31
  • Develop and Run Scala Programs using HBase as Database
    08:24

  • Overview of Data Modeling in HBase
    10:54
  • Develop applications for thin schema
    12:11
  • Develop applications for thick schema
    04:19
  • Examples of HBase filters
    13:35
  • Develop applications using HBase filters
    03:24

  • Introduction
    01:12
  • Overview of Ingestion Technologies
    07:03
  • Overview of Real Time Processing
    05:00
  • Overview of Databases and Visualization
    04:30
  • Overview of Streaming Pipeline Frameworks
    04:15
  • Spark Structured Streaming - Introduction
    09:58
  • Setup Spark Locally
    05:58
  • Development Life Cycle and Setup Project - StreamingDemo
    03:27
  • Define Dependencies
    04:24
  • Externalize Properties
    06:06
  • Define Problem Statement and Understand Data
    02:32
  • Data Processing using Data Frame Operations - Quick Review
    08:19
  • Spark Structured Streaming - Getting Streaming Department Traffic using socket
    15:28
  • Build jar and Deploy - Get Streaming Department Traffic using socket
    12:49
  • Setup Project for Kafka and Spark Structured Streaming Integration
    07:41
  • Kafka and Spark Structured Streaming Integration - using REPL
    14:43
  • Kafka and Spark Structured Streaming Integration - using IDE
    08:56

Instructors

Durga Viswanatha Raju Gadiraju
Technology Adviser and Evangelist
Durga Viswanatha Raju Gadiraju
  • 4.2 Instructor Rating
  • 9,031 Reviews
  • 166,960 Students
  • 18 Courses

13+ years of experience in executing complex projects using vast array of technologies including Big Data and Cloud.

ITVersity, Inc. - a US based organization to provide quality training for IT professionals and we have the track record of training hundreds of thousands of professionals globally.

Building IT career for people with required tools such as high quality material, labs, live support etc to upskill and cross skill is paramount for our organization.

At this time our training offerings are focused on following areas:

* Application Development using Python and SQL

* Big Data and Business Intelligence

* Cloud

* Datawarehousing, Databases

Itversity Support
Support Account for ITVersity Courses.
Itversity Support
  • 4.2 Instructor Rating
  • 8,389 Reviews
  • 151,189 Students
  • 15 Courses

We have built a team to support going forward. If you send messages to this account for our courses, they will be sent to our Helpdesk from where we will be rewriting to our team.

Kamlesh Patidar
KP
  • 4.2 Instructor Rating
  • 281 Reviews
  • 1,747 Students
  • 2 Courses

  • Udemy for Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Featured courses
Udemy
© 2021 Udemy, Inc.