Apache Kafka Fundamentals

1h 4m 52s
English
Paid
In this course, you will master the basic knowledge necessary for a confident start with **Apache Kafka**. You will learn how to configure a message queue, write producers and consumers, and understand how Kafka fits into the architecture of data and event processing platforms. After completing this course, you will easily be able to work with Kafka and understand the operation of similar cloud tools.
Read more about the course

1. Fundamentals of Kafka and Message Queues

You will understand what Kafka is and how it is used in stream and event processing systems. You will learn about the key components of Kafka: topics, messages, consumer groups, and how they interact with each other. You will also learn how a message queue works, how data is written and read from it, and why message order and delivery guarantees are important.

2. Apache Kafka Architecture

We will delve into the key elements of Kafka's architecture. You will learn what topic partitions are, how they relate to brokers, and how data processing takes place within Kafka. I will explain what Zookeeper is, the role it plays, and how it interacts with Kafka brokers and metadata.

3. Setting Up the Development Environment

You will learn how to run Kafka on a Windows environment using Docker. I will provide a step-by-step guide on how to set up a Bitnami Kafka Docker container, and give practical tips for successful installation and launching of the environment.

4. Practicing with Kafka

You will set up your own Kafka topic, master basic commands to work with it. You will also create a producer to write messages and a consumer to read them. We will test their operation using Python and learn to manage consumer offsets using the offset checker.

5. Kafka in Data Processing Platforms

In conclusion, we will explore how Kafka can be integrated into a Data Science platform. You will see three practical scenarios of using Kafka:

  • ETL ingest pipeline
  • Multiple consumer processes
  • Multistage stream processing

These examples will help you start implementing Kafka in your everyday work today.

Watch Online Apache Kafka Fundamentals

Join premium to watch
Go to premium
# Title Duration
1 Introduction 02:16
2 What is Kafka 09:15
3 Basic Kafka Parts 04:25
4 Message Queue Basics 07:39
5 Topics Partitions & Brokers 02:16
6 Brokers & Zookeeper 04:40
7 Development Environment 02:42
8 Bitnami Docker Setup 03:34
9 Basic Topic Commands 04:16
10 Kafka Producer 05:56
11 Kafka Consumer 01:35
12 Testing Producer & Consumer 02:45
13 Working with Consumer Offsets 06:46
14 Examples How Kafka Fits in Data Platforms 05:25
15 Conclusion 01:22

Similar courses to Apache Kafka Fundamentals

The Data Science Course: Complete Data Science Bootcamp 2023

The Data Science Course: Complete Data Science Bootcamp 2023udemy

Category: Data processing and analysis
Duration 31 hours 14 minutes 14 seconds
Build a Large Language Model (From Scratch)

Build a Large Language Model (From Scratch)Sebastian Raschka

Category: Data processing and analysis
Duration
Business Intelligence with Excel

Business Intelligence with Excelzerotomastery.io

Category: Data processing and analysis
Duration 7 hours 41 minutes 24 seconds
Machine Learning A-Z : Become Kaggle Master

Machine Learning A-Z : Become Kaggle Masterudemy

Category: Python, Data processing and analysis
Duration 36 hours 23 minutes 54 seconds
Data Engineering on Azure

Data Engineering on AzureKristijan Bakarić

Category: Azure, Data processing and analysis
Duration 1 hour 20 minutes 57 seconds
Data Analysis with Pandas and Python

Data Analysis with Pandas and Pythonudemy

Category: Python, Data processing and analysis
Duration 19 hours 5 minutes 40 seconds
Build Fast Masterclass

Build Fast MasterclassBuildFast Academy

Category: Python, Data processing and analysis
Duration 7 hours 22 minutes 11 seconds
Machine Learning with Javascript

Machine Learning with JavascriptudemyStephen Grider

Category: Java, Data processing and analysis
Duration 17 hours 42 minutes 20 seconds
Introduction to Data Engineering 2025

Introduction to Data Engineering 2025Andreas Kretz

Category: Data processing and analysis
Duration 44 minutes 26 seconds
Learning Apache Spark

Learning Apache SparkAndreas Kretz

Category: Data processing and analysis
Duration 1 hour 44 minutes 4 seconds