CF

Fundamentals of Apache Airflow

2h 21m 18s
English
Paid

Enhance your data orchestration skills with our practical course onApache Airflow. Begin your journey from the basics and progress towards building real-world orchestration scenarios, including task retries, integration with Spark, and loading external data.

Why Apache Airflow?

While moving data from point A to point B is crucial, ensuring that data is delivered accurately, reliably, and automatically is where Apache Airflow excels. This course will demonstrate how Airflow can transform chaotic, manually configured pipelines into well-organized workflows.

Course Curriculum

Understanding Apache Airflow Architecture

Start with a detailed understanding of the architecture of Airflow and its key components. Lay a solid foundation to build on more advanced concepts.

Advanced Techniques and Features

Master critical techniques such as:

  • Setting up retries to ensure task completion
  • Handling failures gracefully to maintain workflow integrity
  • Utilizing sensors for effective monitoring and control
  • Working with Apache Spark for enhanced data processing
  • Automatically loading data from external sources into a data lake

Who Should Enroll?

This course is perfectly suited for:

  • Beginner data engineers seeking foundational knowledge
  • Experienced professionals aiming to refine their orchestration skills

Equip yourself with practical tools to create scalable and reliable data processing systems using Apache Airflow.

Additional

https://github.com/mushketyk/ztm-data-engineering/tree/main/04-orchestration-with-airflow

About the Author: Zero To Mastery

Zero To Mastery thumbnail

Zero To Mastery (ZTM) is a Toronto-based online coding academy founded by Andrei Neagoie, originally a senior developer at large Canadian tech firms before turning to teaching full-time. The academy's signature is the cohort-based bootcamp track combined with a deep self-paced course library, all aimed at career-changers and self-taught developers preparing to land software-engineering roles at top companies.

The instructor roster has grown well beyond Andrei to include other senior practitioners: Daniel Bourke (machine learning), Aleksa Tešić (DevOps), Jacinto Wong, and others. Courses cover the full software-engineering career path: web development with React and Next.js, Python, machine learning and deep learning, DevOps and cloud, system design, mobile, and the algorithm / data-structure interview prep that gates engineering jobs.

The CourseFlix listing under this source carries over 120 ZTM courses spanning that full range. Material is paid; ZTM itself runs on a monthly / annual membership model. The teaching style favours long-form, project-based courses where students build complete portfolio-quality applications rather than disconnected feature tutorials.

Watch Online 27 lessons

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 27 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Introduction
All Course Lessons (27)
#Lesson TitleDurationAccess
1
Introduction Demo
07:20
2
What Is Apache Airflow?
05:19
3
Airflow’s Architecture
03:15
4
[Optional] What Is a Virtualenv?
06:37
5
[Optional] What Is Docker?
11:03
6
Installing Spark
05:51
7
Installing Airflow
06:33
8
Defining an Airflow DAG
08:03
9
Errors Handling
03:38
10
Idempotent Tasks
04:54
11
Creating a DAG - Part 1
04:58
12
Creating a DAG - Part 2
04:42
13
Handling Failed Tasks
04:09
14
[Exercise] Data Validation
04:31
15
[Exercise] Data Validation - Solution
03:27
16
Spark with Airflow
03:02
17
Using Spark with Airflow - Part 1
07:39
18
Using Spark with Airflow - Part 2
05:52
19
Sensors In Airflow
04:46
20
Using File Sensors
04:08
21
Data Ingestion
05:50
22
Reading Data From Postgres - Part 1
06:03
23
Reading Data from Postgres - Part 2
05:40
24
[Exercise] Average Customer Review
03:53
25
[Exercise] Average Customer Review - Solution
04:33
26
Advanced DAGs
04:26
27
Let's Keep Learning Together!
01:06
Unlock unlimited learning

Get instant access to all 26 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Related courses

Frequently asked questions

What is Fundamentals of Apache Airflow about?
Enhance your data orchestration skills with our practical course on Apache Airflow . Begin your journey from the basics and progress towards building real-world orchestration scenarios, including task retries, integration with Spark, and…
Who teaches this course?
It is taught by Zero To Mastery. You can find more courses by this instructor on the corresponding source page.
How long is the course?
It contains 27 lessons with a total runtime of 2 hours 21 minutes. Every lesson is available to watch online at your own pace.
Is it free to watch?
It is part of CourseFlix's premium catalog. A subscription unlocks the full video player; the course description, table of contents, and preview information are available to everyone.
Where can I watch it online?
The course is available to watch online on CourseFlix at https://courseflix.net/course/fundamentals-of-apache-airflow. The page hosts every lesson with the integrated video player; no download is required.