CF

Azure Data Pipelines with Terraform

4h 20m 29s
English
Paid
Updated September 2026

Azure Data Pipelines with Terraform is a 39-lesson 4 hours 20 minutes self-paced course by Andreas Kretz. Azure is becoming a default choice for companies already running on Microsoft 365, which makes pairing it with Terraform for infrastructure automation a valuable combination for any data engineer.

Course facts

Lessons
39
Duration
4 hours 20 minutes
Level
All levels
Language
English
Updated
2026-09-11
Instructor
Andreas Kretz
Price
Premium

Azure is becoming a default choice for companies already running on Microsoft 365, which makes pairing it with Terraform for infrastructure automation a valuable combination for any data engineer. This course, led by Andreas Kretz, builds that combination through one hands-on project.

The Project

Across 39 lessons you build a full ETL pipeline: pulling data from an external API, processing it with Azure Data Factory and Synapse Analytics, and preparing it for visualization in Power BI. Along the way you implement a Lakehouse setup using the Bronze, Silver, and Gold layers of the Medallion architecture.

Infrastructure as Code

  • Installing and configuring Terraform for Azure, including setting up a Service Principal for secure automated deployments
  • Writing modular, reusable Terraform code to deploy Data Factory, Data Lake Storage, and Synapse Analytics
  • Connecting Data Factory to a live external API to load raw data into the Bronze layer
  • Applying CI/CD principles with Azure DevOps so deployments stay repeatable and stable

By the end you'll have a working, portfolio-ready pipeline and a solid grasp of managing Azure infrastructure as code.

Additional

https://github.com/team-data-science/azure-terraform

Who teaches Azure Data Pipelines with Terraform? Andreas Kretz

Andreas Kretz thumbnail

Andreas Kretz is a German data engineer and one of the most widely followed independent voices on data engineering as a career discipline. He runs the Plumbers of Data Science brand and has been publishing tutorial material continuously since the field consolidated around the modern lake-house stack (Spark, Kafka, Snowflake, Databricks, Airflow).

His CourseFlix listing is the largest single-author catalog under this source — over thirty courses spanning data-pipeline construction, streaming architectures, the cloud-native data stack on AWS / Azure / GCP, the Python and Scala tooling that dominates the field, and the soft-skills / career side of breaking into data engineering. Material is paid and aimed at engineers transitioning into data work or already-working data engineers picking up specific tools.

What lessons are included in Azure Data Pipelines with Terraform?

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 39 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Introduction
All Course Lessons (39)
#Lesson TitleDurationAccess
1
Introduction Demo
01:52
2
Software Setup
04:32
3
Introduction to Azure
01:44
4
Managing Azure
10:52
5
Introduction to Terraform
02:38
6
Terraform Setup on Azure
03:49
7
Terraform Project Structure
06:44
8
Terraform Commands
09:03
9
Backend Deployment
01:40
10
Terraform Modules
09:39
11
Service Principle Deployment
05:18
12
Why CI/CD
05:18
13
CI/CD Process Basics
04:55
14
CI/CD Steps
05:28
15
CI/CD Workflow Example
05:24
16
CI/CD Bascis Summary
01:23
17
Azure CI/CD Pipelines Terminology
10:22
18
Single YAML Pipeline Approach
07:31
19
Azure Dev Ops & Azure Cloud setup
08:27
20
CI/CD Pipeline Implementation
11:58
21
Pipeline Source Code explained & Job Analysis
14:08
22
Executing the CI/CD Pipeline
02:20
23
API Introduction
11:03
24
Azure Data Factory Introduction
05:39
25
Azure Data Factory Components
04:05
26
Working with Data Factory - 1
04:47
27
Working with Data Factory - 2
08:00
28
Working with Data Factory - 3
10:38
29
Working with Data Factory - 4
08:43
30
Introduction to Databricks
06:27
31
Databricks Infrastructure Setup - 1
11:03
32
Databricks Infrastructure Setup - 2
04:22
33
Databricks Infrastructure Setup - 3
04:52
34
The Databricks User Interface
08:10
35
End-To-End Pipeline Execution - 1
05:28
36
End-To-End Pipeline Execution - 2
04:34
37
End-To-End Pipeline Execution - 3
15:09
38
End-To-End Pipeline Execution - 4
06:41
39
End-To-End Pipeline Execution - 5
05:43
Unlock unlimited learning

Get instant access to all 38 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

What courses are similar to Azure Data Pipelines with Terraform?

More courses by Andreas Kretz

Frequently asked questions

What are the prerequisites for enrolling in this course?
Before enrolling, it's beneficial to have a basic understanding of cloud platforms, especially Azure, as the course involves working with Azure Data Factory, Synapse Analytics, and other Azure services. Familiarity with data engineering concepts and some experience with Terraform or infrastructure as code (IaC) will also help you follow along more effectively.
What practical skills will I gain from completing this course?
You will learn how to build a fully automated ETL process using Azure tools, implement Lakehouse and Medallion architecture to optimize data pipelines, and manage infrastructure with Terraform. These skills will be reinforced through a practical project that includes using Azure Data Factory, Synapse Analytics, and Power BI.
Who is the target audience for this course?
The course is designed for data engineers and IT professionals who want to enhance their skills in cloud-based data processing and infrastructure automation using Azure and Terraform. It's particularly suited for those working within the Microsoft365 ecosystem or looking to expand their capabilities in data pipeline construction.
How does the scope of this course compare to other data engineering courses?
This course offers a focused exploration of building data pipelines in Azure using Terraform. Unlike broader data engineering courses, it specifically teaches the integration of Azure services like Data Factory and Synapse Analytics with Terraform's infrastructure automation capabilities, providing a project-based learning experience.
What specific tools and platforms will I use in this course?
The course utilizes Azure's Data Factory, Synapse Analytics, and Power BI for data processing and visualization. You'll also work extensively with Terraform for infrastructure management and Azure DevOps for continuous integration and deployment pipelines.
What topics are not covered in this course?
The course does not cover non-Azure cloud platforms or non-Terraform infrastructure as code tools like AWS or Ansible. It also doesn't delve into non-data engineering aspects of Azure, such as Azure Machine Learning or Azure Kubernetes Service.
How much time should I expect to commit to this course?
The course consists of 39 lessons that cover various components of building and managing data pipelines in Azure. While the total runtime is not specified, expect to spend several hours on video content and additional time on practical exercises and project work.