Apache Kafka Series - Learn Apache Kafka for Beginners v3

8h 20m 45s
English
Paid
April 3, 2024

Welcome to the Apache Kafka Series! Join a community of 20,000+ students learning Kafka. Apache Kafka has become the leading distributed data streaming enterprise  big data technology. Kafka is used in production by over 33% of the Fortune 500 companies such as Netflix, Airbnb, Uber, Walmart and LinkedIn. To learn Kafka easily, step-by-step, you have come to the right place! No prior Kafka knowledge is required

More

If you look at the documentation, you can see that Apache Kafka is not easy to learn...

Thanks to my several years of experience in Kafka and Big Data, I wanted to make learning Kafka accessible to everyone.

We'll take a step-by-step approach to learn all the fundamentals of Apache Kafka.
At the end of this course, you'll be productive and you'll know the following:

  • The Apache Kafka Ecosystem Architecture

  • The Kafka Core Concepts: Topics, Partitions, Brokers, Replicas, Producers, Consumers, and more!

  • Launch your own Kafka cluster in no time using native Kafka binaries – Windows / MacOS X / Linux 

  • Learn and Practice using the Kafka Command Line Interface (CLI)

  • Code Producer and Consumers using the Java API 

  • Real world project using Twitter as a source of data for a producer and ElasticSearch as a sink for our consumer

  • Overview of Advanced APIs (Kafka Connect, Kafka Streams)

  • Real World Case Studies and Big  Use cases

  • Overview of Advanced Kafka for Administrators

  • Advanced Topic Configurations

  • Annexes (starting a Kafka cluster locally, using Docker, etc...)

Note: The hands-on section is based on Java, which is the native Kafka programming language. But, good news! Your learning in Java will be completely applicable to other programming languages, such as Python, C#, Node.js or Scala, and Big Data frameworks such as Spark, NiFi or Akka

This is the course that could improve your career!  

Apache Kafka is a skill in high demand and there are not enough people to fulfil all the open positions. You can boost your income, take on new roles and fun challenges. Many of my students are now the Kafka experts of their companies!

I hope to see you inside the course!

Note: Looking for more advanced Kafka concepts? There are many volumes in the Apache Kafka Series:

  • Learn Kafka for Beginners v2 (this course - great to start)

  • Kafka Connect Hands On Learning

  • Kafka Streams for Data Processing

  • Kafka Cluster Setup & Administration

  • Confluent Schema Registry & Kafka REST Proxy

  • Kafka Security (SSL SASL ACL)

  • Kafka Monitoring and Operations

Happy learning!

Requirements:
  • A recent Windows / Mac / Linux machine with minimum 4GB of RAM, 5 GB of disk space
  • Some understanding of Java Programming
  • Good to have knowledge about Linux command line
  • Desire to learn something awesome and new!
Who this course is for:
  • Developers who want to learn the Apache Kafka Fundamentals, start a cluster and write their first application
  • Architects who want to understand how Apache Kafka fits into their solution architecture
  • Anyone looking to learn the full theory of how Apache Kafka works as a distributed system

What you'll learn:

  • Understand Apache Kafka Ecosystem, Architecture, Core Concepts and Operations
  • Master Concepts such as Topics, Partitions, Brokers, Producers, Consumers
  • Start a personal Kafka development environment
  • Learn major CLIs: kafka-topics, kafka-console-producer, kafka-console-consumer, kafka-consumer-groups, kafka-configs
  • Create your Producers and Consumers in Java to interact with Kafka
  • Program a Real World Twitter Producer & ElasticSearch Consumer
  • Extended APIs Overview (Kafka Connect, Kafka Streams), Case Studies and Big Data Architecture
  • Practice and Understand Log Compaction

Watch Online Apache Kafka Series - Learn Apache Kafka for Beginners v3

Join premium to watch
Go to premium
# Title Duration
1 Course Introduction 02:23
2 Apache Kafka in 5 minutes 05:20
3 Course Objectives 03:58
4 Welcome! - About your instructor 02:46
5 Topics, Partitions and Offsets 06:11
6 Producers and Message Keys 07:25
7 Consumers & Deserialization 04:02
8 Consumer Groups & Consumer Offsets 07:05
9 Brokers and Topics 04:29
10 Topic Replication 05:31
11 Producer Acknowledgements & Topic Durability 02:10
12 Zookeeper 05:15
13 Kafka KRaft - Removing Zookeeper 01:35
14 Theory Roundup 01:34
15 Important: Starting Kafka & Lectures Order 04:07
16 Starting Kafka with Conduktor - Multi Platform 02:11
17 Mac OS X - Download and Setup Kafka in PATH 06:38
18 Mac OS X - Start Zookeeper and Kafka 03:50
19 Mac OS X - Using brew 04:09
20 Linux - Download and Setup Kafka in PATH 07:33
21 Linux - Start Zookeeper and Kafka 03:47
22 Windows WSL2 - Download Kafka and PATH Setup 08:04
23 Windows WSL2 - Start Zookeeper & Kafka 03:21
24 Windows WSL2 - How to Fix Problems 05:28
25 Windows non-WSL2 - Start Zookeeper and Kafka 08:32
26 Mac OS X - Start Kafka in KRaft mode 03:47
27 Linux - Start Kafka in KRaft mode 03:13
28 Windows WSL2 - Start Kafka in KRaft mode 03:05
29 CLI Introduction 03:03
30 Kafka Topics CLI 07:16
31 Kafka Console Producer CLI 06:54
32 Kafka Console Consumer CLI 05:58
33 Kafka Consumers in Group 08:05
34 Kafka Consumer Groups CLI 06:09
35 Resetting Offsets 04:17
36 Conduktor - Demo 05:02
37 Kafka SDK List 01:15
38 Creating Kafka Project 08:38
39 Java Producer 11:06
40 Java Producer Callbacks 12:19
41 Java Producer with Keys 04:40
42 Java Consumer 12:15
43 Java Consumer - Graceful Shutdown 06:56
44 Java Consumer inside Consumer Group 06:02
45 Java Consumer Incremental Cooperative Rebalance & Static Group Membership 07:14
46 Java Consumer Incremental Cooperative Rebalance - Practice 04:54
47 Java Consumer Auto Offset Commit Behavior 03:18
48 Programming - Advanced Tutorials 01:37
49 Real World Project Overview 01:59
50 Wikimedia Producer Project Setup 06:26
51 Wikimedia Producer Implementation 11:55
52 Wikimedia Producer Run 04:45
53 Wikimedia Producer - Producer Config Intros 00:46
54 Producer Acknowledgements Deep Dive 08:49
55 Producer Retries 03:04
56 Idempotent Producer 02:53
57 Safe Kafka Producer Settings 02:00
58 Wikimedia Producer Safe Producer Implementation 04:05
59 Kafka Message Compression 04:49
60 linger.ms and batch.size Producer settings 03:24
61 Wikimedia Producer High Throughput Implementation 03:18
62 Producer Default Partitioner & Sticky Partitioner 04:19
63 [Advanced] max.block.ms and buffer.memory 02:40
64 OpenSearch Consumer - Project Overview 00:53
65 OpenSearch Consumer - Project Setup 03:33
66 Setting up OpenSearch on Docker 02:39
67 Setting up OpenSearch on the Cloud 01:58
68 OpenSearch 101 04:27
69 OpenSearch Consumer Implementation - Part 1 07:27
70 OpenSearch Consumer Implementation Part 2 10:06
71 Consumer Delivery Semantics 03:18
72 OpenSearch Consumer Implementation Part 3 - Idempotence 05:55
73 Consumer Offsets Commit Strategies 04:39
74 OpenSearch Consumer Implementation Part 3 - Delivery Semantics 04:58
75 OpenSearch Consumer Implementation Part 5 - Batching Data 04:20
76 Consumer Offset Reset Behavior 02:07
77 OpenSearch Consumer Implementation Part 6 - Replaying Data 02:13
78 Consumer Internal Threads 05:01
79 Consumer Replica Fetching - Rack Awareness 02:51
80 Kafka Extended APIs - Overview 01:24
81 Kafka Connect Introduction 02:33
82 Kafka Connect Wikimedia & ElasticSearch Hands On 10:58
83 Kafka Streams Introduction 01:53
84 Kafka Streams Hands-On 05:09
85 Kafka Schema Registry Introduction 04:29
86 Kafka Schema Registry Hands On 07:14
87 Which Kafka API should I use? 01:28
88 Choosing Partition Count & Replication Factor 05:22
89 Kafka Topics Naming Convention 01:32
90 Case Study - MovieFlix 05:11
91 Case Study - GetTaxi 04:19
92 Case Study - MySocialMedia 05:33
93 Case Study - MyBank 03:42
94 Case Study - Big Data Ingestion 01:37
95 Case Study - Logging and Metrics Aggregation 01:09
96 Kafka Cluster Setup High Level Architecture Overview 02:56
97 Kafka Monitoring & Operations 02:40
98 Kafka Security 05:52
99 Kafka Multi Cluster & MirrorMaker 04:21
100 Advertised Listeners: Kafka Client & Server Communication Protocol 03:57
101 Changing a Topic Configuration 04:37
102 Segment and Indexes 04:05
103 Log Cleanup Policies 02:54
104 Log Cleanup Delete 02:29
105 Log Compaction Theory 04:53
106 Log Compaction Practice 04:50
107 Unclean Leader Election 01:44
108 Large Messages in Kafka 02:50
109 What's Next? 01:27
110 THANK YOU! 01:33

Similar courses to Apache Kafka Series - Learn Apache Kafka for Beginners v3

Web API From Start to Finish

Web API From Start to Finishiamtimcorey.com (Tim Corey)

Duration 17 hours 59 minutes 54 seconds
PostgreSQL Replication, High Availability HA and Scalability
Node, SQL, & PostgreSQL - Mastering Backend Web Development
DuckDB - The Ultimate Guide

DuckDB - The Ultimate Guideudemy

Duration 5 hours 56 minutes 13 seconds
Complete ASP.NET MVC 5

Complete ASP.NET MVC 5udemy

Duration 7 hours 33 minutes 52 seconds
Mastering Postgres | The most comprehensive course on PostgreSQL
Fundamentals of Backend Communications and Protocols

Fundamentals of Backend Communications and Protocolsudemy

Duration 15 hours 35 minutes 37 seconds
Introduction to Microsoft Orleans
Fullstack Typescript with TailwindCSS and tRPC Using Modern Features of PostgreSQL
Clean Code: Writing Code for Humans

Clean Code: Writing Code for Humans pluralsight

Duration 3 hours 10 minutes 36 seconds