Stream big data in real time with Spark and integrate any data source, from Kafka to Twitter.
Spark Streaming with Scala
11h 17m 52s
English
Paid
Nothing static, all in motion.
Your probably know by now: Spark is the most popular computing engine for big data, the most maintained, and with a proven track record of performance. It's 100 times faster than the old MapReduce paradigm, and can easily be extended with machine learning and streaming capabilities, and much more.
In this course, we'll take the natural step forward: process big data as it arrives.
About the Author: rockthejvm.com
Rock the JVM aims to be the go-to resource for learning Scala, Akka, Spark and everything JVM. Everything is battle-tested on yours truly. I'm Daniel Ciocîrlan, a developer and teacher from Bucharest, Romania. I've started Rock the JVM to share everything that I learned through pain, blood, sweat and tears and distill everything in action-packed courses with a smooth learning curve. Since 2013, I've been teaching a variety of programming topics to people all over the world and at every level of the game, from senior engineers processing trillions of data points to 7-year-olds moving spaceships on the screen.
At first, I mastered Java, then I taught it for 3.5 years to university students. I found Scala 7 years ago when a good friend told me about functional collections. I was blown away, and still am to this day as I discover more of the Scala magic. The rabbit hole only went deeper from there: I learned Akka with my own small projects, then Spark and big data while at Palantir. I left Palantir in mid 2019, but it's very likely my old pipelines are still crunching data every day, as we speak.
I hope you enjoy my material - in the meantime, I've just started on Twitter and would love to meet you!
A bit more
I spend most of my time between Scala, Akka and Spark, and I'm really fortunate I get to work with them and teach them to other developers. I love the expressiveness and the concise power of a good technology. I learn from the community and the people I've had in my courses and I love to share back what I've learned. I love to speak at meetups and conferences and I adore people's faces when something really hard "clicks" in their mind and their face lights up. I don't know how people react to my online courses in real time, but I love the "this changed my life" comments.
I've graduated from University Politehnica of Bucharest with a BSc and a MSc in Computer Science. My Bachelor and Master theses were on quantum computation, of all things. That can be explained by the fact that I was really into physics before - I was winning medals at international physics competitions.
When I'm not coding, I dance salsa and bachata and I tend to play an absurd amount of Ed Sheeran on my acoustic guitar.
Watch Online 31 lessons
0:00
/ #1: Welcome
All Course Lessons (31)
| # | Lesson Title | Duration | Access |
|---|---|---|---|
| 1 | Welcome Demo | 18:05 | |
| 2 | Scala Recap | 25:06 | |
| 3 | Spark Recap | 27:11 | |
| 4 | Spark Streaming First Principles | 08:29 | |
| 5 | Streaming DataFrames | 27:17 | |
| 6 | Streaming Aggregations | 15:58 | |
| 7 | Streaming Joins | 25:24 | |
| 8 | Streaming Datasets | 26:46 | |
| 9 | Discretized Streams (DStreams) | 31:57 | |
| 10 | DStreams Transformations | 28:40 | |
| 11 | DStreams Window Functions | 32:11 | |
| 12 | Kafka & Structured Streaming | 25:08 | |
| 13 | Kafka & DStreams | 29:57 | |
| 14 | JDBC with Postgres | 11:51 | |
| 15 | Akka and Akka Streams | 28:58 | |
| 16 | Cassandra | 21:55 | |
| 17 | Setting up a Twitter App | 08:17 | |
| 18 | Our First Custom Receiver | 15:45 | |
| 19 | Reading Tweets | 20:05 | |
| 20 | Reading Tweets: Exercises | 15:22 | |
| 21 | Sentiment Analysis on Tweets using NLP | 20:50 | |
| 22 | Event Time Windows | 31:20 | |
| 23 | Event Time Windows: Exercises | 14:54 | |
| 24 | Processing Time Windows | 11:16 | |
| 25 | Watermarks | 25:16 | |
| 26 | Watermarks, Part 2 | 25:22 | |
| 27 | Arbitrary Stateful Computation | 25:40 | |
| 28 | Arbitrary Stateful Computation | 16:36 | |
| 29 | Setting up the REST Server and the Kafka Broker | 27:51 | |
| 30 | Integrating Spark Structured Streaming, Test, Run! | 33:54 | |
| 31 | You Rock! | 00:31 |
Unlock unlimited learning
Get instant access to all 30 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.
Learn more about subscription