CourseFlix

Fundamentals of Apache Airflow

2h 21m 18s
English
Paid

Enhance your data orchestration skills with our practical course onApache Airflow. Begin your journey from the basics and progress towards building real-world orchestration scenarios, including task retries, integration with Spark, and loading external data.

Why Apache Airflow?

While moving data from point A to point B is crucial, ensuring that data is delivered accurately, reliably, and automatically is where Apache Airflow excels. This course will demonstrate how Airflow can transform chaotic, manually configured pipelines into well-organized workflows.

Course Curriculum

Understanding Apache Airflow Architecture

Start with a detailed understanding of the architecture of Airflow and its key components. Lay a solid foundation to build on more advanced concepts.

Advanced Techniques and Features

Master critical techniques such as:

  • Setting up retries to ensure task completion
  • Handling failures gracefully to maintain workflow integrity
  • Utilizing sensors for effective monitoring and control
  • Working with Apache Spark for enhanced data processing
  • Automatically loading data from external sources into a data lake

Who Should Enroll?

This course is perfectly suited for:

  • Beginner data engineers seeking foundational knowledge
  • Experienced professionals aiming to refine their orchestration skills

Equip yourself with practical tools to create scalable and reliable data processing systems using Apache Airflow.

Additional

https://github.com/mushketyk/ztm-data-engineering/tree/main/04-orchestration-with-airflow

About the Author: Zero To Mastery

Zero To Mastery thumbnail

Zero To Mastery (ZTM) is a Toronto-based online coding academy founded by Andrei Neagoie, originally a senior developer at large Canadian tech firms before turning to teaching full-time. The academy's signature is the cohort-based bootcamp track combined with a deep self-paced course library, all aimed at career-changers and self-taught developers preparing to land software-engineering roles at top companies.

The instructor roster has grown well beyond Andrei to include other senior practitioners: Daniel Bourke (machine learning), Aleksa Tešić (DevOps), Jacinto Wong, and others. Courses cover the full software-engineering career path: web development with React and Next.js, Python, machine learning and deep learning, DevOps and cloud, system design, mobile, and the algorithm / data-structure interview prep that gates engineering jobs.

The CourseFlix listing under this source carries over 120 ZTM courses spanning that full range. Material is paid; ZTM itself runs on a monthly / annual membership model. The teaching style favours long-form, project-based courses where students build complete portfolio-quality applications rather than disconnected feature tutorials.

Watch Online 27 lessons

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 27 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Introduction
All Course Lessons (27)
#Lesson TitleDurationAccess
1
Introduction Demo
07:20
2
What Is Apache Airflow?
05:19
3
Airflow’s Architecture
03:15
4
[Optional] What Is a Virtualenv?
06:37
5
[Optional] What Is Docker?
11:03
6
Installing Spark
05:51
7
Installing Airflow
06:33
8
Defining an Airflow DAG
08:03
9
Errors Handling
03:38
10
Idempotent Tasks
04:54
11
Creating a DAG - Part 1
04:58
12
Creating a DAG - Part 2
04:42
13
Handling Failed Tasks
04:09
14
[Exercise] Data Validation
04:31
15
[Exercise] Data Validation - Solution
03:27
16
Spark with Airflow
03:02
17
Using Spark with Airflow - Part 1
07:39
18
Using Spark with Airflow - Part 2
05:52
19
Sensors In Airflow
04:46
20
Using File Sensors
04:08
21
Data Ingestion
05:50
22
Reading Data From Postgres - Part 1
06:03
23
Reading Data from Postgres - Part 2
05:40
24
[Exercise] Average Customer Review
03:53
25
[Exercise] Average Customer Review - Solution
04:33
26
Advanced DAGs
04:26
27
Let's Keep Learning Together!
01:06
Unlock unlimited learning

Get instant access to all 26 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Course content

27 lessons · 2h 21m 18s
Show all 27 lessons
  1. 1 Introduction 07:20
  2. 2 What Is Apache Airflow? 05:19
  3. 3 Airflow’s Architecture 03:15
  4. 4 [Optional] What Is a Virtualenv? 06:37
  5. 5 [Optional] What Is Docker? 11:03
  6. 6 Installing Spark 05:51
  7. 7 Installing Airflow 06:33
  8. 8 Defining an Airflow DAG 08:03
  9. 9 Errors Handling 03:38
  10. 10 Idempotent Tasks 04:54
  11. 11 Creating a DAG - Part 1 04:58
  12. 12 Creating a DAG - Part 2 04:42
  13. 13 Handling Failed Tasks 04:09
  14. 14 [Exercise] Data Validation 04:31
  15. 15 [Exercise] Data Validation - Solution 03:27
  16. 16 Spark with Airflow 03:02
  17. 17 Using Spark with Airflow - Part 1 07:39
  18. 18 Using Spark with Airflow - Part 2 05:52
  19. 19 Sensors In Airflow 04:46
  20. 20 Using File Sensors 04:08
  21. 21 Data Ingestion 05:50
  22. 22 Reading Data From Postgres - Part 1 06:03
  23. 23 Reading Data from Postgres - Part 2 05:40
  24. 24 [Exercise] Average Customer Review 03:53
  25. 25 [Exercise] Average Customer Review - Solution 04:33
  26. 26 Advanced DAGs 04:26
  27. 27 Let's Keep Learning Together! 01:06

Related courses

  • Introduction to Data Engineering 2025 thumbnail

    Introduction to Data Engineering 2025

    By: Andreas Kretz
    Welcome to your comprehensive introduction to Data Engineering, a foundational course designed to enhance your understanding of this pivotal field and the.
    44 minutes 26 seconds 5 / 5
  • Modern Data Warehouses & Data Lakes thumbnail

    Modern Data Warehouses & Data Lakes

    By: Andreas Kretz
    As a data engineer, being adept in working with analytical platforms is crucial.
    58 minutes 9 seconds 5 / 5
  • TensorFlow Developer Certificate in 2023: Zero to Mastery thumbnail

    TensorFlow Developer Certificate in 2023: Zero to Mastery

    By: Zero To Mastery
    Learn TensorFlow. Pass the TensorFlow Developer Certificate Exam. Get Hired as a TensorFlow developer. This course will take you from a TensorFlow beginner to b
    62 hours 43 minutes 54 seconds

Frequently asked questions

What is Fundamentals of Apache Airflow about?
Enhance your data orchestration skills with our practical course on Apache Airflow . Begin your journey from the basics and progress towards building real-world orchestration scenarios, including task retries, integration with Spark, and…
Who teaches Fundamentals of Apache Airflow?
Fundamentals of Apache Airflow is taught by Zero To Mastery. You can find more courses by this instructor on the corresponding source page.
How long is Fundamentals of Apache Airflow?
Fundamentals of Apache Airflow contains 27 lessons with a total runtime of 2 hours 21 minutes. All lessons are available to watch online at your own pace.
Is Fundamentals of Apache Airflow free to watch?
Fundamentals of Apache Airflow is part of CourseFlix's premium catalog. A CourseFlix subscription unlocks the full video player; the course description, table of contents, and preview information are available to everyone.
Where can I watch Fundamentals of Apache Airflow online?
Fundamentals of Apache Airflow is available to watch online on CourseFlix at https://courseflix.net/course/fundamentals-of-apache-airflow. The page hosts every lesson with the integrated video player; no download is required.