CF

Apache Kafka Series - Learn Apache Kafka for Beginners v3

8h 20m 45s
English
Paid

Learn Apache Kafka with clear steps and simple terms. This guide helps you build skill with real tasks and hands‑on work. You will learn how Kafka moves data, how apps use it, and how you can run it on your own machine.

What You Will Learn

You will start with the core ideas in Kafka. You will then set up a local cluster and use the command line tools. After that, you will write simple apps that read and write data to Kafka.

  • The Kafka ecosystem and core parts
  • Topics, partitions, brokers, replicas, producers, and consumers
  • How to start a Kafka cluster on Windows, macOS, or Linux
  • How to use key CLI tools
  • How to write simple producer and consumer code in Java
  • A hands‑on project with Twitter data and Elasticsearch
  • Overviews of Kafka Connect and Kafka Streams
  • Admin tasks and topic settings
  • Extra guides for local tests and Docker use

About the Hands‑On Work

You will write your apps in Java. Kafka uses Java at its core, so this helps you learn faster. The ideas you learn will also help you work with Python, C#, Node.js, Scala, or tools like Spark, NiFi, and Akka.

Course Series

This course is part of a larger Kafka series. Here are the related topics you can explore:

  • Kafka for Beginners
  • Kafka Connect
  • Kafka Streams
  • Kafka Cluster Setup and Admin
  • Schema Registry and REST Proxy
  • Kafka Security
  • Kafka Monitoring and Ops

Requirements

  • A Windows, macOS, or Linux machine with at least 4 GB RAM and 5 GB free space
  • Basic Java knowledge
  • Basic Linux command line skill is helpful
  • A clear goal to learn Kafka

Who This Course Is For

  • Developers who want to learn Kafka and build simple apps
  • Architects who want to see how Kafka fits into a system design
  • Learners who want to understand Kafka as a full distributed system

Skills You Will Build

  • Kafka architecture and core ideas
  • How core parts like producers and consumers work
  • How to start a local Kafka setup
  • How to use tools like kafka-topics and kafka-console-producer
  • How to write Java apps that send and read data
  • How to code a Twitter producer and Elasticsearch consumer
  • How to use Kafka Connect and Kafka Streams at a high level
  • How log compaction works

About the Author: Udemy

Udemy thumbnail

Udemy is the largest open marketplace for online courses on the internet. Founded in 2010 by Eren Bali, Oktay Caglar, and Gagan Biyani and headquartered in San Francisco, the company went public on the Nasdaq in 2021 under the ticker UDMY. The platform hosts well over two hundred thousand courses across software development, IT and cloud, data science, design, business, marketing, and creative skills, taught by tens of thousands of independent instructors. Roughly seventy million learners use it worldwide, and the corporate arm — Udemy Business — supplies a curated subset of that catalog to enterprise customers.

Because Udemy is a marketplace rather than a single editorial publisher, the catalog is uneven by design. The strongest material lives in the long-form, project-based courses authored by working engineers — full-stack JavaScript, React, Node.js, Python data science, AWS, Docker and Kubernetes, mobile development with Flutter and React Native, and cloud certification preparation. The CourseFlix listing under this source is the slice of that catalog that has been mirrored here for offline-friendly viewing, organized by topic and updated as new releases land. Pricing on Udemy itself swings dramatically with the site's near-permanent sales, which is why the platform is best treated as a deep reference catalog: pick instructors with strong reviews and a track record of updating their material rather than buying on the headline price alone.

Watch Online 110 lessons

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 110 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Course Introduction
All Course Lessons (110)
#Lesson TitleDurationAccess
1
Course Introduction Demo
02:23
2
Apache Kafka in 5 minutes
05:20
3
Course Objectives
03:58
4
Welcome! - About your instructor
02:46
5
Topics, Partitions and Offsets
06:11
6
Producers and Message Keys
07:25
7
Consumers & Deserialization
04:02
8
Consumer Groups & Consumer Offsets
07:05
9
Brokers and Topics
04:29
10
Topic Replication
05:31
11
Producer Acknowledgements & Topic Durability
02:10
12
Zookeeper
05:15
13
Kafka KRaft - Removing Zookeeper
01:35
14
Theory Roundup
01:34
15
Important: Starting Kafka & Lectures Order
04:07
16
Starting Kafka with Conduktor - Multi Platform
02:11
17
Mac OS X - Download and Setup Kafka in PATH
06:38
18
Mac OS X - Start Zookeeper and Kafka
03:50
19
Mac OS X - Using brew
04:09
20
Linux - Download and Setup Kafka in PATH
07:33
21
Linux - Start Zookeeper and Kafka
03:47
22
Windows WSL2 - Download Kafka and PATH Setup
08:04
23
Windows WSL2 - Start Zookeeper & Kafka
03:21
24
Windows WSL2 - How to Fix Problems
05:28
25
Windows non-WSL2 - Start Zookeeper and Kafka
08:32
26
Mac OS X - Start Kafka in KRaft mode
03:47
27
Linux - Start Kafka in KRaft mode
03:13
28
Windows WSL2 - Start Kafka in KRaft mode
03:05
29
CLI Introduction
03:03
30
Kafka Topics CLI
07:16
31
Kafka Console Producer CLI
06:54
32
Kafka Console Consumer CLI
05:58
33
Kafka Consumers in Group
08:05
34
Kafka Consumer Groups CLI
06:09
35
Resetting Offsets
04:17
36
Conduktor - Demo
05:02
37
Kafka SDK List
01:15
38
Creating Kafka Project
08:38
39
Java Producer
11:06
40
Java Producer Callbacks
12:19
41
Java Producer with Keys
04:40
42
Java Consumer
12:15
43
Java Consumer - Graceful Shutdown
06:56
44
Java Consumer inside Consumer Group
06:02
45
Java Consumer Incremental Cooperative Rebalance & Static Group Membership
07:14
46
Java Consumer Incremental Cooperative Rebalance - Practice
04:54
47
Java Consumer Auto Offset Commit Behavior
03:18
48
Programming - Advanced Tutorials
01:37
49
Real World Project Overview
01:59
50
Wikimedia Producer Project Setup
06:26
51
Wikimedia Producer Implementation
11:55
52
Wikimedia Producer Run
04:45
53
Wikimedia Producer - Producer Config Intros
00:46
54
Producer Acknowledgements Deep Dive
08:49
55
Producer Retries
03:04
56
Idempotent Producer
02:53
57
Safe Kafka Producer Settings
02:00
58
Wikimedia Producer Safe Producer Implementation
04:05
59
Kafka Message Compression
04:49
60
linger.ms and batch.size Producer settings
03:24
61
Wikimedia Producer High Throughput Implementation
03:18
62
Producer Default Partitioner & Sticky Partitioner
04:19
63
[Advanced] max.block.ms and buffer.memory
02:40
64
OpenSearch Consumer - Project Overview
00:53
65
OpenSearch Consumer - Project Setup
03:33
66
Setting up OpenSearch on Docker
02:39
67
Setting up OpenSearch on the Cloud
01:58
68
OpenSearch 101
04:27
69
OpenSearch Consumer Implementation - Part 1
07:27
70
OpenSearch Consumer Implementation Part 2
10:06
71
Consumer Delivery Semantics
03:18
72
OpenSearch Consumer Implementation Part 3 - Idempotence
05:55
73
Consumer Offsets Commit Strategies
04:39
74
OpenSearch Consumer Implementation Part 3 - Delivery Semantics
04:58
75
OpenSearch Consumer Implementation Part 5 - Batching Data
04:20
76
Consumer Offset Reset Behavior
02:07
77
OpenSearch Consumer Implementation Part 6 - Replaying Data
02:13
78
Consumer Internal Threads
05:01
79
Consumer Replica Fetching - Rack Awareness
02:51
80
Kafka Extended APIs - Overview
01:24
81
Kafka Connect Introduction
02:33
82
Kafka Connect Wikimedia & ElasticSearch Hands On
10:58
83
Kafka Streams Introduction
01:53
84
Kafka Streams Hands-On
05:09
85
Kafka Schema Registry Introduction
04:29
86
Kafka Schema Registry Hands On
07:14
87
Which Kafka API should I use?
01:28
88
Choosing Partition Count & Replication Factor
05:22
89
Kafka Topics Naming Convention
01:32
90
Case Study - MovieFlix
05:11
91
Case Study - GetTaxi
04:19
92
Case Study - MySocialMedia
05:33
93
Case Study - MyBank
03:42
94
Case Study - Big Data Ingestion
01:37
95
Case Study - Logging and Metrics Aggregation
01:09
96
Kafka Cluster Setup High Level Architecture Overview
02:56
97
Kafka Monitoring & Operations
02:40
98
Kafka Security
05:52
99
Kafka Multi Cluster & MirrorMaker
04:21
100
Advertised Listeners: Kafka Client & Server Communication Protocol
03:57
101
Changing a Topic Configuration
04:37
102
Segment and Indexes
04:05
103
Log Cleanup Policies
02:54
104
Log Cleanup Delete
02:29
105
Log Compaction Theory
04:53
106
Log Compaction Practice
04:50
107
Unclean Leader Election
01:44
108
Large Messages in Kafka
02:50
109
What's Next?
01:27
110
THANK YOU!
01:33
Unlock unlimited learning

Get instant access to all 109 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Related courses

Frequently asked questions

What are the prerequisites for this course?
This course requires basic knowledge of Java and basic Linux command line experience. Additionally, you will need a machine running Windows, macOS, or Linux with at least 4 GB of RAM and 5 GB of free disk space to set up and operate your local Kafka cluster.
What hands-on projects will I work on during the course?
The course features a hands-on project that involves working with Twitter data and Elasticsearch. You will learn to set up a Kafka producer to stream data, which will then be processed and visualized using Elasticsearch.
Who is the target audience for this course?
The course is designed for beginners who want to learn Apache Kafka from scratch. It is particularly suited for software developers and data engineers interested in building real-time data pipelines and streaming applications.
How does the depth of this course compare to similar courses?
This course offers an introduction to the core ideas of Kafka and covers setting up a local cluster, using command line tools, and writing basic producer and consumer applications. It also provides overviews of Kafka Connect and Kafka Streams, making it suitable for beginners looking for a foundational understanding.
What specific tools or platforms will I use in this course?
You will use various tools and platforms such as Apache Kafka, Zookeeper, and the command line interface for managing Kafka topics and consumers. The course also includes using Docker for local tests and guides for starting Kafka in KRaft mode.
What topics are not covered in this course?
The course does not cover advanced Kafka topics like Kafka Security, Kafka Monitoring and Operations, or the Schema Registry and REST Proxy in detail. These are likely addressed in other courses within the Kafka series.
What is the expected time commitment for this course?
The course consists of 110 lessons. While the total runtime is not specified, students should expect to spend several hours on the video content and additional time on hands-on exercises and projects.