Fundamentals of Apache Spark and PySpark
Course description
Apache Spark is one of the most important tools for any aspiring Data Engineer or Data Scientist. And PySpark is a way to unleash the full power of Spark using familiar Python.
This course is designed for those who want to confidently enter the world of big data. We will explore the architecture of Spark, teach you how to write clear and efficient code in PySpark, and show you how to create scalable data processing pipelines.
The training is practice-based: you will work with real datasets, solve practical tasks, and acquire skills that are truly in demand by employers.
If you want to learn how to analyze massive amounts of data, quickly clean and transform information, and use the tools utilized by Netflix, Amazon, and other industry leaders — this course is just for you.
Watch Online
All Course Lessons (29)
| # | Lesson Title | Duration | Access |
|---|---|---|---|
| 1 | Introduction Demo | 07:30 | |
| 2 | [Optional] What Is a Virtualenv? | 06:37 | |
| 3 | Apache Spark | 03:44 | |
| 4 | How Spark Works | 04:24 | |
| 5 | Spark Application | 07:41 | |
| 6 | DataFrames | 06:43 | |
| 7 | Installing Spark | 05:51 | |
| 8 | Inside Airbnb Data | 07:02 | |
| 9 | Writing Your First Spark Job | 07:05 | |
| 10 | Lazy Processing | 02:16 | |
| 11 | [Exercise] Basic Functions | 01:29 | |
| 12 | [Exercise] Basic Functions - Solution | 06:41 | |
| 13 | Aggregating Data | 04:00 | |
| 14 | Joining Data | 04:40 | |
| 15 | Aggregations and Joins with Spark | 06:10 | |
| 16 | Complex Data Types | 05:09 | |
| 17 | [Exercise] Aggregate Functions | 00:50 | |
| 18 | [Exercise] Aggregate Functions - Solution | 05:54 | |
| 19 | User Defined Functions | 03:25 | |
| 20 | Data Shuffle | 06:14 | |
| 21 | Data Accumulators | 03:42 | |
| 22 | Optimizing Spark Jobs | 07:39 | |
| 23 | Submitting Spark Jobs | 04:29 | |
| 24 | Other Spark APIs | 05:16 | |
| 25 | Spark SQL | 04:33 | |
| 26 | [Exercise] Advanced Spark | 02:10 | |
| 27 | [Exercise] Advanced Spark - Solution | 05:26 | |
| 28 | Summary | 03:08 | |
| 29 | Let's Keep Learning Together! | 01:06 |
Unlock unlimited learning
Get instant access to all 28 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.
Learn more about subscriptionComments
0 commentsWant to join the conversation?
Sign in to commentSimilar courses
Introduction to Data Engineering
Time Series Analysis, Forecasting, and Machine Learning
Data Engineering on AWS
Introduction to Data Engineering 2025
