CourseFlix

Apache Kafka Series - Learn Apache Kafka for Beginners v3

8h 20m 45s
English
Paid

Learn Apache Kafka with clear steps and simple terms. This guide helps you build skill with real tasks and hands‑on work. You will learn how Kafka moves data, how apps use it, and how you can run it on your own machine.

What You Will Learn

You will start with the core ideas in Kafka. You will then set up a local cluster and use the command line tools. After that, you will write simple apps that read and write data to Kafka.

  • The Kafka ecosystem and core parts
  • Topics, partitions, brokers, replicas, producers, and consumers
  • How to start a Kafka cluster on Windows, macOS, or Linux
  • How to use key CLI tools
  • How to write simple producer and consumer code in Java
  • A hands‑on project with Twitter data and Elasticsearch
  • Overviews of Kafka Connect and Kafka Streams
  • Admin tasks and topic settings
  • Extra guides for local tests and Docker use

About the Hands‑On Work

You will write your apps in Java. Kafka uses Java at its core, so this helps you learn faster. The ideas you learn will also help you work with Python, C#, Node.js, Scala, or tools like Spark, NiFi, and Akka.

Course Series

This course is part of a larger Kafka series. Here are the related topics you can explore:

  • Kafka for Beginners
  • Kafka Connect
  • Kafka Streams
  • Kafka Cluster Setup and Admin
  • Schema Registry and REST Proxy
  • Kafka Security
  • Kafka Monitoring and Ops

Requirements

  • A Windows, macOS, or Linux machine with at least 4 GB RAM and 5 GB free space
  • Basic Java knowledge
  • Basic Linux command line skill is helpful
  • A clear goal to learn Kafka

Who This Course Is For

  • Developers who want to learn Kafka and build simple apps
  • Architects who want to see how Kafka fits into a system design
  • Learners who want to understand Kafka as a full distributed system

Skills You Will Build

  • Kafka architecture and core ideas
  • How core parts like producers and consumers work
  • How to start a local Kafka setup
  • How to use tools like kafka-topics and kafka-console-producer
  • How to write Java apps that send and read data
  • How to code a Twitter producer and Elasticsearch consumer
  • How to use Kafka Connect and Kafka Streams at a high level
  • How log compaction works

About the Author: udemy

udemy thumbnail

Udemy is a global online marketplace for teaching and learning. It connects students with instructors who share skills in many fields.

Who Uses Udemy

Millions of learners use Udemy to gain skills for work and personal growth. Companies, governments, and nonprofits also use Udemy to train their teams.

What Udemy Offers

Udemy provides a curated set of business and technical courses. These courses help teams build practical skills and support ongoing learning at work.

Watch Online 110 lessons

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 110 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Course Introduction
All Course Lessons (110)
#Lesson TitleDurationAccess
1
Course Introduction Demo
02:23
2
Apache Kafka in 5 minutes
05:20
3
Course Objectives
03:58
4
Welcome! - About your instructor
02:46
5
Topics, Partitions and Offsets
06:11
6
Producers and Message Keys
07:25
7
Consumers & Deserialization
04:02
8
Consumer Groups & Consumer Offsets
07:05
9
Brokers and Topics
04:29
10
Topic Replication
05:31
11
Producer Acknowledgements & Topic Durability
02:10
12
Zookeeper
05:15
13
Kafka KRaft - Removing Zookeeper
01:35
14
Theory Roundup
01:34
15
Important: Starting Kafka & Lectures Order
04:07
16
Starting Kafka with Conduktor - Multi Platform
02:11
17
Mac OS X - Download and Setup Kafka in PATH
06:38
18
Mac OS X - Start Zookeeper and Kafka
03:50
19
Mac OS X - Using brew
04:09
20
Linux - Download and Setup Kafka in PATH
07:33
21
Linux - Start Zookeeper and Kafka
03:47
22
Windows WSL2 - Download Kafka and PATH Setup
08:04
23
Windows WSL2 - Start Zookeeper & Kafka
03:21
24
Windows WSL2 - How to Fix Problems
05:28
25
Windows non-WSL2 - Start Zookeeper and Kafka
08:32
26
Mac OS X - Start Kafka in KRaft mode
03:47
27
Linux - Start Kafka in KRaft mode
03:13
28
Windows WSL2 - Start Kafka in KRaft mode
03:05
29
CLI Introduction
03:03
30
Kafka Topics CLI
07:16
31
Kafka Console Producer CLI
06:54
32
Kafka Console Consumer CLI
05:58
33
Kafka Consumers in Group
08:05
34
Kafka Consumer Groups CLI
06:09
35
Resetting Offsets
04:17
36
Conduktor - Demo
05:02
37
Kafka SDK List
01:15
38
Creating Kafka Project
08:38
39
Java Producer
11:06
40
Java Producer Callbacks
12:19
41
Java Producer with Keys
04:40
42
Java Consumer
12:15
43
Java Consumer - Graceful Shutdown
06:56
44
Java Consumer inside Consumer Group
06:02
45
Java Consumer Incremental Cooperative Rebalance & Static Group Membership
07:14
46
Java Consumer Incremental Cooperative Rebalance - Practice
04:54
47
Java Consumer Auto Offset Commit Behavior
03:18
48
Programming - Advanced Tutorials
01:37
49
Real World Project Overview
01:59
50
Wikimedia Producer Project Setup
06:26
51
Wikimedia Producer Implementation
11:55
52
Wikimedia Producer Run
04:45
53
Wikimedia Producer - Producer Config Intros
00:46
54
Producer Acknowledgements Deep Dive
08:49
55
Producer Retries
03:04
56
Idempotent Producer
02:53
57
Safe Kafka Producer Settings
02:00
58
Wikimedia Producer Safe Producer Implementation
04:05
59
Kafka Message Compression
04:49
60
linger.ms and batch.size Producer settings
03:24
61
Wikimedia Producer High Throughput Implementation
03:18
62
Producer Default Partitioner & Sticky Partitioner
04:19
63
[Advanced] max.block.ms and buffer.memory
02:40
64
OpenSearch Consumer - Project Overview
00:53
65
OpenSearch Consumer - Project Setup
03:33
66
Setting up OpenSearch on Docker
02:39
67
Setting up OpenSearch on the Cloud
01:58
68
OpenSearch 101
04:27
69
OpenSearch Consumer Implementation - Part 1
07:27
70
OpenSearch Consumer Implementation Part 2
10:06
71
Consumer Delivery Semantics
03:18
72
OpenSearch Consumer Implementation Part 3 - Idempotence
05:55
73
Consumer Offsets Commit Strategies
04:39
74
OpenSearch Consumer Implementation Part 3 - Delivery Semantics
04:58
75
OpenSearch Consumer Implementation Part 5 - Batching Data
04:20
76
Consumer Offset Reset Behavior
02:07
77
OpenSearch Consumer Implementation Part 6 - Replaying Data
02:13
78
Consumer Internal Threads
05:01
79
Consumer Replica Fetching - Rack Awareness
02:51
80
Kafka Extended APIs - Overview
01:24
81
Kafka Connect Introduction
02:33
82
Kafka Connect Wikimedia & ElasticSearch Hands On
10:58
83
Kafka Streams Introduction
01:53
84
Kafka Streams Hands-On
05:09
85
Kafka Schema Registry Introduction
04:29
86
Kafka Schema Registry Hands On
07:14
87
Which Kafka API should I use?
01:28
88
Choosing Partition Count & Replication Factor
05:22
89
Kafka Topics Naming Convention
01:32
90
Case Study - MovieFlix
05:11
91
Case Study - GetTaxi
04:19
92
Case Study - MySocialMedia
05:33
93
Case Study - MyBank
03:42
94
Case Study - Big Data Ingestion
01:37
95
Case Study - Logging and Metrics Aggregation
01:09
96
Kafka Cluster Setup High Level Architecture Overview
02:56
97
Kafka Monitoring & Operations
02:40
98
Kafka Security
05:52
99
Kafka Multi Cluster & MirrorMaker
04:21
100
Advertised Listeners: Kafka Client & Server Communication Protocol
03:57
101
Changing a Topic Configuration
04:37
102
Segment and Indexes
04:05
103
Log Cleanup Policies
02:54
104
Log Cleanup Delete
02:29
105
Log Compaction Theory
04:53
106
Log Compaction Practice
04:50
107
Unclean Leader Election
01:44
108
Large Messages in Kafka
02:50
109
What's Next?
01:27
110
THANK YOU!
01:33
Unlock unlimited learning

Get instant access to all 109 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription