CF

Fundamentals of Apache Kafka

2h 33m 26s
English
Paid

Learn the Basics of Apache Kafka from Scratch and Master Building Reliable, Scalable Real-time Data Processing Systems.

Course Overview

In this course, you will become familiar with Kafka's architecture, the principles of producers and consumers, ensuring reliability of stream processing, message delivery semantics, and tools from the Kafka ecosystem—such as Kafka Connect and Schema Registry.

Apache Kafka is at the core of real-time data processing systems used by the most innovative companies in the world. If you want to understand how massive data streams are processed "on the fly," this course will be an excellent starting point.

What You Will Learn

Understanding Kafka Architecture

Gain practical insights into how Kafka is structured and why this is important. You'll explore the system's architecture and its key components.

Building Producers and Consumers

Learn to create robust producers and consumers for data streams, crucial for processing real-time information effectively.

Data Processing and Delivery Methods

Understand the trade-offs between different data delivery methods and how Kafka ensures reliable data processing.

Tools in the Kafka Ecosystem

  • Kafka Connect: Discover how to integrate Kafka with other systems using connectors.
  • Schema Registry: Learn about managing and validating data formats in Kafka pipelines.

Course Outcomes

By the end of the course, you won't just understand how Kafka works—you will be able to use it to build reliable, scalable real-time data processing systems, equipped to handle real-world challenges.

Additional

https://github.com/mushketyk/ztm-data-engineering/tree/main/07-kafka-streaming

About the Author: Zero To Mastery

Zero To Mastery thumbnail

Zero To Mastery (ZTM) is a Toronto-based online coding academy founded by Andrei Neagoie, originally a senior developer at large Canadian tech firms before turning to teaching full-time. The academy's signature is the cohort-based bootcamp track combined with a deep self-paced course library, all aimed at career-changers and self-taught developers preparing to land software-engineering roles at top companies.

The instructor roster has grown well beyond Andrei to include other senior practitioners: Daniel Bourke (machine learning), Aleksa Tešić (DevOps), Jacinto Wong, and others. Courses cover the full software-engineering career path: web development with React and Next.js, Python, machine learning and deep learning, DevOps and cloud, system design, mobile, and the algorithm / data-structure interview prep that gates engineering jobs.

The CourseFlix listing under this source carries over 120 ZTM courses spanning that full range. Material is paid; ZTM itself runs on a monthly / annual membership model. The teaching style favours long-form, project-based courses where students build complete portfolio-quality applications rather than disconnected feature tutorials.

Watch Online 26 lessons

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 26 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Introduction
All Course Lessons (26)
#Lesson TitleDurationAccess
1
Introduction Demo
08:21
2
What Is Apache Kafka?
07:00
3
[Optional] What Is a Virtualenv?
06:37
4
[Optional] What Is Docker?
11:03
5
Partitioning Data
08:56
6
Kafka API
07:42
7
Kafka Architecture
03:15
8
Set Up Kafka
05:53
9
Writing to Kafka
06:07
10
Reading from Kafka
07:37
11
Data Durability
06:39
12
Kafka vs Queues
02:11
13
[Exercise] Processing Records
03:44
14
[Exercise] Processing Records - Solution
02:59
15
Delivery Semantics
05:53
16
Kafka Transactions
04:34
17
Log Compaction
03:23
18
Kafka Connect
06:59
19
Using Kafka Connect
09:44
20
Outbox Pattern
04:31
21
Schema Registry
08:01
22
Using Schema Registry
08:10
23
Tiered Storage
03:28
24
[Exercise] Track Order Status Changes
04:27
25
[Exercise] Track Order Status Changes - Solution
05:06
26
Let's Keep Learning Together!
01:06
Unlock unlimited learning

Get instant access to all 25 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Related courses

Frequently asked questions

What prerequisites should I have before enrolling in this course?
Before enrolling, it is beneficial to have a basic understanding of distributed systems and familiarity with programming in a language like Java or Python. While the course offers optional lessons on Virtualenv and Docker, prior knowledge of these tools can be helpful for setting up the development environment.
What projects or practical exercises are included in the course?
The course includes practical exercises such as 'Processing Records' and 'Track Order Status Changes'. These exercises are designed to reinforce the concepts of data streaming and processing using Kafka, and solutions are provided for better understanding.
Who is the target audience for this course?
This course is intended for software engineers, data engineers, and IT professionals who are interested in learning about real-time data processing systems. It is also suitable for anyone looking to understand the architecture and functioning of Kafka, including those new to the Kafka ecosystem.
How does this course compare in depth and scope to other Kafka courses?
The course focuses on foundational elements of Kafka, including its architecture, producers, consumers, and ecosystem tools like Kafka Connect and Schema Registry. It provides a balanced overview suitable for beginners, and may not cover advanced topics like Kafka Streams or detailed performance tuning, which might be found in more advanced courses.
What tools and platforms will I learn to use in this course?
You'll learn to use Kafka Connect and Schema Registry, important tools in the Kafka ecosystem. Kafka Connect facilitates the integration of Kafka with various systems, while Schema Registry helps manage and validate data formats across Kafka pipelines.
Are there any topics that are not covered in this course?
The course does not cover advanced topics such as Kafka Streams or in-depth performance tuning. It also does not delve into the management of Kafka clusters in production environments, focusing instead on foundational concepts and tools.
What is the expected time commitment for completing the course?
The course consists of 26 lessons and is designed to be completed at your own pace. Although the total runtime isn't specified, students should expect to dedicate several hours to both the instructional content and practical exercises to fully grasp the material.