Fundamentals of Apache Spark and PySpark

2h 20m 54s
English
Paid

Course description

Apache Spark is one of the most important tools for any aspiring Data Engineer or Data Scientist. And PySpark is a way to unleash the full power of Spark using familiar Python.

This course is designed for those who want to confidently enter the world of big data. We will explore the architecture of Spark, teach you how to write clear and efficient code in PySpark, and show you how to create scalable data processing pipelines.

The training is practice-based: you will work with real datasets, solve practical tasks, and acquire skills that are truly in demand by employers.

If you want to learn how to analyze massive amounts of data, quickly clean and transform information, and use the tools utilized by Netflix, Amazon, and other industry leaders — this course is just for you.

Watch Online

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 29 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Introduction

All Course Lessons (29)

#Lesson TitleDurationAccess
1
Introduction Demo
07:30
2
[Optional] What Is a Virtualenv?
06:37
3
Apache Spark
03:44
4
How Spark Works
04:24
5
Spark Application
07:41
6
DataFrames
06:43
7
Installing Spark
05:51
8
Inside Airbnb Data
07:02
9
Writing Your First Spark Job
07:05
10
Lazy Processing
02:16
11
[Exercise] Basic Functions
01:29
12
[Exercise] Basic Functions - Solution
06:41
13
Aggregating Data
04:00
14
Joining Data
04:40
15
Aggregations and Joins with Spark
06:10
16
Complex Data Types
05:09
17
[Exercise] Aggregate Functions
00:50
18
[Exercise] Aggregate Functions - Solution
05:54
19
User Defined Functions
03:25
20
Data Shuffle
06:14
21
Data Accumulators
03:42
22
Optimizing Spark Jobs
07:39
23
Submitting Spark Jobs
04:29
24
Other Spark APIs
05:16
25
Spark SQL
04:33
26
[Exercise] Advanced Spark
02:10
27
[Exercise] Advanced Spark - Solution
05:26
28
Summary
03:08
29
Let's Keep Learning Together!
01:06

Unlock unlimited learning

Get instant access to all 28 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Comments

0 comments

Want to join the conversation?

Sign in to comment

Similar courses

Machine Learning Design Questions

Machine Learning Design Questions

Sources: algoexpert
Several videos presenting the practice required to pass machine learning design interviews. It will not be boring.
3 hours 3 minutes 57 seconds
Machine Learning with Python : COMPLETE COURSE FOR BEGINNERS

Machine Learning with Python : COMPLETE COURSE FOR BEGINNERS

Sources: udemy
Machine Learning and artificial intelligence (AI) is everywhere; if you want to know how companies like Google, Amazon, and even Udemy extract meaning and insig
13 hours 12 minutes 31 seconds
Apache Kafka Fundamentals

Apache Kafka Fundamentals

Sources: Andreas Kretz
In this course, you will acquire the basic knowledge necessary for confidently starting to work with Apache Kafka. You will learn how to set up a message...
1 hour 4 minutes 52 seconds
Data Engineering on AWS

Data Engineering on AWS

Sources: Andreas Kretz
This course is the perfect start for those who want to learn cloud technologies and start working with Amazon Web Services (AWS), one of the most popular..
4 hours 46 minutes 38 seconds
Introduction to Data Engineering 2025

Introduction to Data Engineering 2025

Sources: Andreas Kretz
This introductory course will help you better understand what data engineering is and what role a data engineer plays in the field of Data Science. To begin...
44 minutes 26 seconds