Apache Kafka Series - Kafka Streams for Data Processing
More
Kafka Streams is the easiest way to write your applications on top of Kafka:
> Easiest way to transform your data using the High Level DSL
> Exactly Once semantics support out of the box!
> Deploy and Scale your Kafka Streams application without a cluster!
> Perform Aggregations, joins, and any operations you may think of using only a few lines of code!
> Built on top of Kafka, for fault tolerance, scalability and resiliency
Instructor
Stephane Maarek is the instructor of this course. He is a Kafka Expert, and the author of the highly-rated Apache Kafka Series on Udemy, having taught already to 40000+ students and received 12000+ reviews.
He shares all his Kafka knowledge on the platform, taking the time to explain every concept and provide students with both theoretical and practical dimensions. You are in good hands!
Hands On & Theory Based Course
Note: This course is based on Java 8, and will include one example in Scala. Kafka Streams is Java based, and therefore is not suited for any other programming language.
Each section can be either theoretical, or a practice section.
> Through practice, you will be challenged by writing your own Kafka Streams application. The solutions will be thoroughly explained, and you will learn some tips on how to use Kafka Streams the best way.
> Through theory, you will learn about all the available API, the inner workings of the library, as well as some exciting concepts such as Exactly Once Semantics!
This course is the first and only available Kafka Streams course on the web. Get it now to become a Kafka expert!
Section outline:
Kafka Streams - First Look: Let's get Kafka started and run your first Kafka Streams application, WordCount
End-to-End Kafka Streams Application : Write the code for the WordCount, bring in the dependencies, build and package your application, and learn how to scale it. This is a complete end to end example
KStream and KTable Simple Operations: Learn all the stateless operations available for the KStream and KTable API
Practice Exercise - Favourite Colour: Practice your newly acquired skills by writing your own Kafka Streams application, Favourite Colour. It will be challenging! Includes a Scala version of the example
KStream and KTable Advanced Operations: Learn all the stateful operations available for the KStream and KTable API
Exactly Once Semantics - Theory: Learn what EOS (Exactly Once Semantics) is, how Kafka 0.11 enables it, and how to activate them in Kafka Streams
Exactly Once - Practice Exercise - Bank Balance: Practice your newly acquired knowledge by writing your own Kafka Streams Exactly Once application, to compute a running Bank Balance for your customers
Testing your Kafka Streams Application: Learn how to test the WordCount Kafka Streams Topology with Kafka Streams v1.1.0
Learn The Hottest Kafka Data Processing Library Now!
Kafka is increasingly becoming a must-have skill, and this course will set you up for fast success using the Kafka Streams API. The job market will need people with your newly acquired skillset!
Who is the target audience?
This course is for developers and devops who would like to learn how to write, package, deploy and run a Kafka Streams applications
This course is for architects who would like to understand how Kafka Streams work and its position in the Kafka-centered data pipeline and enterprise architecture
You will need some good understanding of Kafka before starting this course
Requirements:
Good knowledge of Apache Kafka
A recent computer (Windows / Linux / Mac OS X) with 4GB of RAM
Good knowledge of Java 8 Programming Language
- Developers and devops who would like to learn how to write, package, deploy and run a Kafka Streams applications
- Architects who would like to understand how Kafka Streams work and its position in the Kafka-centered data pipeline and enterprise architecture
- People who fully understand the basics of Kafka
- Java 8 and Scala developers only
What you'll learn:
- Write four Kafka Streams application in Java 8
- Configure Kafka Streams to use Exactly Once Semantics
- Scale Kafka Streams applications
- Program with the High Level DSL of Kafka Streams
- Build and package your application
- Write tests for your Kafka Streams Topology
- And so much more!
Watch Online Apache Kafka Series - Kafka Streams for Data Processing
| # | Title | Duration |
|---|---|---|
| 1 | What is Kafka Streams? | 04:48 |
| 2 | Course Objective / Prerequisites / Target Students | 04:12 |
| 3 | About your Instructor: Stephane Maarek | 02:21 |
| 4 | Running your first Kafka Streams Application: WordCount | 12:53 |
| 5 | Kafka Streams vs other stream processing libraries (Spark Streaming, NiFI, Flink | 03:02 |
| 6 | Section Objective | 01:46 |
| 7 | Kafka Streams Core Concepts | 03:29 |
| 8 | Environment and IDE Setup: Java 8, Maven, IntelliJ IDEA | 04:04 |
| 9 | Starter Project Setup | 07:58 |
| 10 | Kafka Streams Application Properties | 07:26 |
| 11 | Java 8 Lambda Functions - quick overview | 02:56 |
| 12 | Word Count Application Topology | 14:09 |
| 13 | Printing the Kafka Streams Topology | 01:34 |
| 14 | Kafka Streams Graceful Shutdown | 02:04 |
| 15 | Running Application from IntelliJ IDEA | 09:23 |
| 16 | Debugging Application from IntelliJ IDEA | 04:07 |
| 17 | Internal Topics for our Kafka Streams Application | 03:48 |
| 18 | Packaging the application as Fat Jar & Running the Fat Jar | 04:35 |
| 19 | Scaling our Application | 04:00 |
| 20 | Section Wrap-Up | 01:41 |
| 21 | Section Objectives | 01:15 |
| 22 | KStream & KTables | 03:40 |
| 23 | Stateless vs Stateful Operations | 01:42 |
| 24 | MapValues / Map | 01:36 |
| 25 | Filter / FilterNot | 01:23 |
| 26 | FlatMapValues / FlatMap | 02:11 |
| 27 | Branch | 02:46 |
| 28 | SelectKey | 01:15 |
| 29 | Reading from Kafka | 02:17 |
| 30 | Writing to Kafka | 01:53 |
| 31 | Streams Marked for Re-Partition | 02:53 |
| 32 | Refresher on Log Compaction | 17:46 |
| 33 | KStream and KTables Duality | 02:21 |
| 34 | Transforming a KTable to a KStream | 00:42 |
| 35 | Transforming a KStream to a KTable | 01:25 |
| 36 | Section Summary | 00:48 |
| 37 | FavouriteColour - Practice Exercise Description & Guidance | 03:21 |
| 38 | Stuck? Here are some Hints! | 02:41 |
| 39 | Java Solution | 08:34 |
| 40 | Running the application | 05:47 |
| 41 | Scala Solution | 06:21 |
| 42 | Section Objective | 01:01 |
| 43 | KTable groupBy | 01:47 |
| 44 | KGroupedStream / KGroupedTable Count | 02:06 |
| 45 | KGroupedStream / KGroupedTable Aggregate | 04:13 |
| 46 | KGroupedStream / KGroupedTable Reduce | 01:41 |
| 47 | KStream peek | 02:09 |
| 48 | KStream Transform / TransformValues | 01:13 |
| 49 | What if I want to write to an external System? | 01:41 |
| 50 | Summary Diagram | 01:04 |
| 51 | What's Exactly Once? | 06:08 |
| 52 | Exactly Once in Kafka 0.11 | 02:42 |
| 53 | What's the problem with at least once anyway? | 01:30 |
| 54 | How to do exactly once in Kafka Streams | 02:05 |
| 55 | BankBalance - Exercise Overview | 02:16 |
| 56 | Kafka Producer Guidance | 01:42 |
| 57 | Kafka Producer Solution | 14:18 |
| 58 | Kafka Streams Guidance & Hints | 01:56 |
| 59 | Kafka Streams Solution | 09:13 |
| 60 | Running the BankBalance Application | 04:43 |
| 61 | Section Summary | 00:57 |
| 62 | What are joins in Kafka Streams? | 02:50 |
| 63 | Join Constraints and GlobalKTables | 02:54 |
| 64 | The different types of joins: Inner Join, Left Join, Outer Join | 02:46 |
| 65 | Creating a join with UserEnrich Kafka Streams App | 12:34 |
| 66 | Running the Kafka Streams Join application | 05:08 |
| 67 | Testing in Kafka Streams | 04:03 |
| 68 | Setup your Kafka Streams project | 03:59 |
| 69 | Hands-On: Test your WordCount application | 14:29 |
| 70 | Congratulations and next steps | 02:33 |
| 71 | THANK YOU! | 01:33 |
Similar courses to Apache Kafka Series - Kafka Streams for Data Processing
Apache Kafka Series - Learn Apache Kafka for Beginners v3
Mastering Postgres | The most comprehensive course on PostgreSQL
The Complete 2023 Web Development Bootcamp
Web API From Start to Finish
Database Design & Implementation
Apache Flink
Fundamentals of Database Engineering
Neo4j: GraphDB Foundations with Cypher
High Performance SQLite
