Apache Kafka Fundamentals

1h 4m 52s
English
Paid
In this course, you will master the basic knowledge necessary for a confident start with **Apache Kafka**. You will learn how to configure a message queue, write producers and consumers, and understand how Kafka fits into the architecture of data and event processing platforms. After completing this course, you will easily be able to work with Kafka and understand the operation of similar cloud tools.
Read more about the course

1. Fundamentals of Kafka and Message Queues

You will understand what Kafka is and how it is used in stream and event processing systems. You will learn about the key components of Kafka: topics, messages, consumer groups, and how they interact with each other. You will also learn how a message queue works, how data is written and read from it, and why message order and delivery guarantees are important.

2. Apache Kafka Architecture

We will delve into the key elements of Kafka's architecture. You will learn what topic partitions are, how they relate to brokers, and how data processing takes place within Kafka. I will explain what Zookeeper is, the role it plays, and how it interacts with Kafka brokers and metadata.

3. Setting Up the Development Environment

You will learn how to run Kafka on a Windows environment using Docker. I will provide a step-by-step guide on how to set up a Bitnami Kafka Docker container, and give practical tips for successful installation and launching of the environment.

4. Practicing with Kafka

You will set up your own Kafka topic, master basic commands to work with it. You will also create a producer to write messages and a consumer to read them. We will test their operation using Python and learn to manage consumer offsets using the offset checker.

5. Kafka in Data Processing Platforms

In conclusion, we will explore how Kafka can be integrated into a Data Science platform. You will see three practical scenarios of using Kafka:

  • ETL ingest pipeline
  • Multiple consumer processes
  • Multistage stream processing

These examples will help you start implementing Kafka in your everyday work today.

Watch Online Apache Kafka Fundamentals

Join premium to watch
Go to premium
# Title Duration
1 Introduction 02:16
2 What is Kafka 09:15
3 Basic Kafka Parts 04:25
4 Message Queue Basics 07:39
5 Topics Partitions & Brokers 02:16
6 Brokers & Zookeeper 04:40
7 Development Environment 02:42
8 Bitnami Docker Setup 03:34
9 Basic Topic Commands 04:16
10 Kafka Producer 05:56
11 Kafka Consumer 01:35
12 Testing Producer & Consumer 02:45
13 Working with Consumer Offsets 06:46
14 Examples How Kafka Fits in Data Platforms 05:25
15 Conclusion 01:22

Similar courses to Apache Kafka Fundamentals

PyTorch for Deep Learning

PyTorch for Deep Learningzerotomastery.io

Category: Data processing and analysis
Duration 52 hours 27 seconds
Data Engineering on GCP

Data Engineering on GCPAndreas Kretz

Category: Data processing and analysis
Duration 1 hour 17 minutes 33 seconds
The Data Bootcamp: Transform your Data using dbt™

The Data Bootcamp: Transform your Data using dbt™udemy

Category: Data processing and analysis
Duration 4 hours 10 minutes 51 seconds
Data Platform & Pipeline Design

Data Platform & Pipeline DesignAndreas Kretz

Category: Data processing and analysis
Duration 1 hour 59 minutes 5 seconds
MongoDB Fundamentals

MongoDB FundamentalsAndreas Kretz

Category: MongoDB, Data processing and analysis
Duration 1 hour 23 minutes 19 seconds
Introduction to Data Engineering

Introduction to Data Engineeringzerotomastery.io

Category: Data processing and analysis
Duration 57 minutes 26 seconds
Time Series Analysis, Forecasting, and Machine Learning

Time Series Analysis, Forecasting, and Machine Learningudemy

Category: Python, Data processing and analysis
Duration 22 hours 47 minutes 45 seconds
Data Analysis for Beginners: Python & Statistics

Data Analysis for Beginners: Python & Statisticszerotomastery.io

Category: Python, Data processing and analysis
Duration 6 hours 34 minutes 20 seconds
Streaming with Kafka & Spark

Streaming with Kafka & SparkAndreas Kretz

Category: Data processing and analysis
Duration 2 hours 46 minutes 25 seconds
Introduction to Data Engineering 2025

Introduction to Data Engineering 2025Andreas Kretz

Category: Data processing and analysis
Duration 44 minutes 26 seconds